BaşlarkenLiderlik tablosuDecode calculatorModellerOrganizationsReportsMatrix runsDonanımKarşılaştırmalı DeğerlendirmelerPazar yeriKiralıklarProAPI Dokümanlar
Dil
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 26 of 28.
  • Qwen3.8-27B-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP4+FP86 contexts × 8 concurrency levels
    Peak decode
    2,348.0 tok/s
    Peak prefill
    6,938 tok/s
  • Qwen3.8-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,147.3 tok/s
    Peak prefill
    9,033 tok/s
  • Qwen3.8-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,178.5 tok/s
    Peak prefill
    9,219 tok/s
  • Qwen3.8-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,471.5 tok/s
    Peak prefill
    13K tok/s
  • Qwen3.8-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,172.9 tok/s
    Peak prefill
    13.4K tok/s
  • Qwen3.6-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,236.4 tok/s
    Peak prefill
    35.5K tok/s
  • Qwen3.6-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,623.2 tok/s
    Peak prefill
    37K tok/s
  • Qwen3.6-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,602.8 tok/s
    Peak prefill
    46.5K tok/s
  • Qwen3.6-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,165.8 tok/s
    Peak prefill
    47.7K tok/s
  • Qwen3.6-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,692.5 tok/s
    Peak prefill
    9,618 tok/s
  • Qwen3.6-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,189.5 tok/s
    Peak prefill
    9,781 tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,389.1 tok/s
    Peak prefill
    14.2K tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,412.8 tok/s
    Peak prefill
    14.5K tok/s
  • Qwen3.5-9BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,688.7 tok/s
    Peak prefill
    28.1K tok/s
  • Qwen3.5-9BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,455.0 tok/s
    Peak prefill
    28.9K tok/s
  • Qwen3.5-9B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,710.2 tok/s
    Peak prefill
    28.4K tok/s
  • Qwen3.5-9B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,456.9 tok/s
    Peak prefill
    28.9K tok/s
  • Qwen3.5-4BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    6,715.5 tok/s
    Peak prefill
    43.2K tok/s
  • Qwen3.5-4BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    6,277.1 tok/s
    Peak prefill
    44.5K tok/s
  • Qwen3.5-4B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,252.4 tok/s
    Peak prefill
    43.1K tok/s
  • Qwen3.5-4B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    6,315.6 tok/s
    Peak prefill
    44.3K tok/s
  • Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,747.4 tok/s
    Peak prefill
    36.3K tok/s
  • Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,598.7 tok/s
    Peak prefill
    37K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,629.2 tok/s
    Peak prefill
    27.6K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    6,237.8 tok/s
    Peak prefill
    31.7K tok/s