Per iniziareClassificaDecode calculatorModelliOrganizationsReportsMatrix runsHardwareValutazioni comparativeMarketplaceNoleggiProDocumentazione API
Lingua
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 25 of 28.
  • Qwen3.6-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    698.2 tok/s
    Peak prefill
    9,796 tok/s
  • Qwen3.5-9BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,712.8 tok/s
    Peak prefill
    26.4K tok/s
  • Qwen3.5-4BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,657.1 tok/s
    Peak prefill
    38.2K tok/s
  • Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,945.8 tok/s
    Peak prefill
    32.4K tok/s
  • Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,550.5 tok/s
    Peak prefill
    9,165 tok/s
  • gemma-4-12B-itH200 NVL 141GB · sglang 0.0.0.dev16562+g6283bde44 · BF166 contexts × 8 concurrency levels
    Peak decode
    611.1 tok/s
    Peak prefill
    9,653 tok/s
  • gemma-4-26B-A4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    497.4 tok/s
    Peak prefill
    10.9K tok/s
  • Step-3.7-Flash-NVFP4H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9269+g834400705 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,432.7 tok/s
    Peak prefill
    17.3K tok/s
  • Step-3.7-Flash-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9269+g834400705 · FP86 contexts × 8 concurrency levels
    Peak decode
    889.0 tok/s
    Peak prefill
    24.8K tok/s
  • Qwen3.6-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,622.2 tok/s
    Peak prefill
    37.2K tok/s
  • Qwen3.6-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,167.1 tok/s
    Peak prefill
    47.5K tok/s
  • Qwen3.6-35B-A3B-FP8-dynamicH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,119.1 tok/s
    Peak prefill
    46.7K tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,414.4 tok/s
    Peak prefill
    14.5K tok/s
  • Muse-Glimmer-30BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
    Peak decode
    2,608.7 tok/s
    Peak prefill
    10.4K tok/s
  • Muse-Glimmer-30B-FP8-blockH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP85 contexts × 8 concurrency levels
    Peak decode
    3,896.2 tok/s
    Peak prefill
    15K tok/s
  • Muse-Glimmer-30B-FP8-blockH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP85 contexts × 8 concurrency levels
    Peak decode
    4,216.1 tok/s
    Peak prefill
    15.5K tok/s
  • gemma-4-E4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
    Peak decode
    5,036.5 tok/s
    Peak prefill
    25.3K tok/s
  • Nanbeige4.2-3BH200 NVL 141GB · vllm 0.1.dev20429+g62f6de733 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,776.1 tok/s
    Peak prefill
    28.6K tok/s
  • Nanbeige4.2-3BH200 NVL 141GB · sglang 0.0.0.dev17517+g3e59d89e5 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,195.7 tok/s
    Peak prefill
    26.6K tok/s
  • gemma-4-E2B-itH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
    Peak decode
    11.2K tok/s
    Peak prefill
    33K tok/s
  • gemma-4-26B-A4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,659.8 tok/s
    Peak prefill
    15.8K tok/s
  • gemma-4-26B-A4B-it-FP8-dynamicH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,382.1 tok/s
    Peak prefill
    18.1K tok/s
  • Qwen3.8-27B-NVFP4H200 NVL 141GB · sglang 0.0.0.dev16762+g0c088029c · NVFP4+FP86 contexts × 8 concurrency levels
    Peak decode
    1,559.6 tok/s
    Peak prefill
    6,303 tok/s
  • Qwen3.8-27B-FP8H200 NVL 141GB · sglang 0.0.0.dev16762+g0c088029c · FP86 contexts × 8 concurrency levels
    Peak decode
    1,297.6 tok/s
    Peak prefill
    13.9K tok/s
  • Qwen3.8-Flash-Next-NVFP4H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9023+g73a255206 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    2,235.0 tok/s
    Peak prefill
    19.7K tok/s