Get startedLeaderboardDecode calculatorModelsOrganizationsReportsMatrix runsHardwareBenchmarksMarketplaceRentalsProAPI Docs
Language
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 5 of 28.
  • gemma-4-E2B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    9,005.7 tok/s
    Peak prefill
    40.4K tok/s
  • gemma-4-31B-it-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    858.3 tok/s
    Peak prefill
    8,122 tok/s
  • gemma-4-31B-it-FP8-dynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    710.3 tok/s
    Peak prefill
    7,822 tok/s
  • gemma-4-31B-it-FP8-blockRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    360.4 tok/s
    Peak prefill
    6,106 tok/s
  • gemma-4-26B-A4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,064.5 tok/s
    Peak prefill
    21.5K tok/s
  • gemma-4-26B-A4B-it-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    176.9 tok/s
    Peak prefill
    31.5K tok/s
  • gemma-4-26B-A4B-it-FP8-dynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    384.8 tok/s
    Peak prefill
    34.1K tok/s
  • gemma-4-12B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,570.4 tok/s
    Peak prefill
    11K tok/s
  • gemma-4-12B-it-FP8-DynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    714.8 tok/s
    Peak prefill
    19K tok/s
  • Qwen3.8-27B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · NVFP4+FP86 contexts × 8 concurrency levels
    Peak decode
    885.9 tok/s
    Peak prefill
    12.8K tok/s
  • Qwen3.8-27B-NVFP4-BF16-LMHeadRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · NVFP4+FP86 contexts × 8 concurrency levels
    Peak decode
    1,196.1 tok/s
    Peak prefill
    13.4K tok/s
  • Qwen3.8-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · FP86 contexts × 8 concurrency levels
    Peak decode
    515.0 tok/s
    Peak prefill
    9,770 tok/s
  • Qwen3.6-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,698.2 tok/s
    Peak prefill
    37.1K tok/s
  • Qwen3.6-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,746.3 tok/s
    Peak prefill
    37.6K tok/s
  • Qwen3.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,971.1 tok/s
    Peak prefill
    20.5K tok/s
  • Qwen3.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,431.9 tok/s
    Peak prefill
    21.3K tok/s
  • Qwen3.5-9B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,683.4 tok/s
    Peak prefill
    20.4K tok/s
  • Qwen3.5-9B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,422.0 tok/s
    Peak prefill
    21.4K tok/s
  • Qwen3.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    6,224.8 tok/s
    Peak prefill
    33K tok/s
  • Qwen3.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,354.8 tok/s
    Peak prefill
    33.9K tok/s
  • Qwen3.5-4B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,307.7 tok/s
    Peak prefill
    32.9K tok/s
  • Qwen3.5-4B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,403.3 tok/s
    Peak prefill
    34.5K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
    Peak decode
    2,739.2 tok/s
    Peak prefill
    13.6K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,682.0 tok/s
    Peak prefill
    13.2K tok/s
  • Qwen3.5-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
    Peak decode
    4,907.2 tok/s
    Peak prefill
    37.1K tok/s