Get startedLeaderboardDecode calculatorModelsOrganizationsReportsMatrix runsHardwareBenchmarksMarketplaceRentalsProAPI Docs
Language
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 10 of 15.
  • NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,741.8 tok/s
    Peak prefill
    6,443 tok/s
  • Qwen3.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    769.8 tok/s
    Peak prefill
    1,664 tok/s
  • Qwen3.6-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    823.9 tok/s
    Peak prefill
    1,681 tok/s
  • gemma-3-12b-it-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    372.4 tok/s
    Peak prefill
    4,968 tok/s
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    1,689.9 tok/s
    Peak prefill
    4,990 tok/s
  • Qwen3-Coder-30B-A3B-Instruct-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    1,564.2 tok/s
    Peak prefill
    8,096 tok/s
  • Qwen3.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    796.6 tok/s
    Peak prefill
    1,725 tok/s
  • Qwen3.5-9B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    1,164.5 tok/s
    Peak prefill
    6,836 tok/s
  • granite-4.2-8b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    692.1 tok/s
    Peak prefill
    5,033 tok/s
  • Qwen3.6-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,234.0 tok/s
    Peak prefill
    6,676 tok/s
  • Qwen3-8B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    737.3 tok/s
    Peak prefill
    5,772 tok/s
  • Qwen3.5-9B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    961.2 tok/s
    Peak prefill
    7,105 tok/s
  • Tess-4.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    672.7 tok/s
    Peak prefill
    2,107 tok/s
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    3,160.9 tok/s
    Peak prefill
    10.6K tok/s
  • Qwen3.6-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,936.7 tok/s
    Peak prefill
    6,578 tok/s
  • gemma-4-12B-it-FP8-DynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    935.4 tok/s
    Peak prefill
    3,847 tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    2,080.9 tok/s
    Peak prefill
    9,922 tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,263.4 tok/s
    Peak prefill
    10.6K tok/s
  • Phi-4-reasoning-plus-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    204.6 tok/s
    Peak prefill
    3,317 tok/s
  • Tess-4-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    672.0 tok/s
    Peak prefill
    2,210 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,729.1 tok/s
    Peak prefill
    8,877 tok/s
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    3,409.4 tok/s
    Peak prefill
    4,722 tok/s
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    2,708.2 tok/s
    Peak prefill
    6,871 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,736.7 tok/s
    Peak prefill
    7,524 tok/s
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,095.8 tok/s
    Peak prefill
    8,943 tok/s