Aan de slagLeaderboardDecode calculatorModellenOrganizationsReportsMatrix runsHardwareBenchmarksMarktplaatsVerhuurProAPI Docs
Taal
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 9 of 15.
  • Qwen3.5-4B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    4,738.8 tok/s
    Peak prefill
    11K tok/s
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,499.1 tok/s
    Peak prefill
    3,276 tok/s
  • gemma-3-12b-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,335.5 tok/s
    Peak prefill
    4,882 tok/s
  • Agents-A1-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,155.8 tok/s
    Peak prefill
    11.7K tok/s
  • gemma-4-12B-it-FP8-DynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,388.8 tok/s
    Peak prefill
    3,857 tok/s
  • Qwen3-8B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    1,889.2 tok/s
    Peak prefill
    5,567 tok/s
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,189.3 tok/s
    Peak prefill
    11.1K tok/s
  • Qwen3.5-9B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,915.2 tok/s
    Peak prefill
    6,700 tok/s
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ4 contexts × 8 concurrency levels
    Peak decode
    291.6 tok/s
    Peak prefill
    3,188 tok/s
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,186.7 tok/s
    Peak prefill
    11.8K tok/s
  • Qwen3-14B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    1,436.5 tok/s
    Peak prefill
    3,122 tok/s
  • Qwen3.5-9B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    3,499.4 tok/s
    Peak prefill
    6,902 tok/s
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    546.1 tok/s
    Peak prefill
    3,243 tok/s
  • Trinity-Mini-FP8-BlockCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 block5 contexts × 8 concurrency levels
    Peak decode
    2,422.8 tok/s
    Peak prefill
    10.3K tok/s
  • Ministral-3-3B-Instruct-2512-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,524.0 tok/s
    Peak prefill
    12.3K tok/s
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    574.1 tok/s
    Peak prefill
    3,265 tok/s
  • Phi-4-reasoning-plus-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    1,423.6 tok/s
    Peak prefill
    3,210 tok/s
  • granite-4.2-30b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    282.4 tok/s
    Peak prefill
    2,003 tok/s
  • Qwen3-8B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    580.6 tok/s
    Peak prefill
    5,731 tok/s
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,310.2 tok/s
    Peak prefill
    4,010 tok/s
  • Reka-Flash-3-21B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic3 contexts × 8 concurrency levels
    Peak decode
    1,012.0 tok/s
    Peak prefill
    2,732 tok/s
  • granite-4.2-8b-fp8RTX 3090 24GB · vllm 0.32.0 · FP84 contexts × 8 concurrency levels
    Peak decode
    554.0 tok/s
    Peak prefill
    5,971 tok/s
  • Qwen3.6-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    747.0 tok/s
    Peak prefill
    1,693 tok/s
  • gemma-4-26B-A4B-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,919.3 tok/s
    Peak prefill
    7,265 tok/s
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    2,813.1 tok/s
    Peak prefill
    5,961 tok/s