Erste SchritteLeaderboardDecode calculatorModelleOrganizationsReportsMatrix runsHardwareBenchmarksMarktplatzVermietungenProAPI-Doku
Sprache
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 18 of 28.
  • Qwen3.6-35B-A3B-FP8-dynamicH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,065.3 tok/s
    Peak prefill
    41.4K tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,554.0 tok/s
    Peak prefill
    14.5K tok/s
  • gemma-4-E4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
    Peak decode
    7,744.3 tok/s
    Peak prefill
    40.6K tok/s
  • gemma-4-E2B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
    Peak decode
    13.1K tok/s
    Peak prefill
    62K tok/s
  • gemma-4-31B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
    Peak decode
    616.9 tok/s
    Peak prefill
    7,892 tok/s
  • gemma-4-31B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,143.4 tok/s
    Peak prefill
    4,486 tok/s
  • gemma-4-31B-it-FP8-blockH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,189.1 tok/s
    Peak prefill
    11.4K tok/s
  • gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,389.3 tok/s
    Peak prefill
    40.2K tok/s
  • gemma-4-26B-A4B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    4,355.9 tok/s
    Peak prefill
    24.7K tok/s
  • gemma-4-12B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,491.4 tok/s
    Peak prefill
    19.4K tok/s
  • gemma-4-12B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,875.8 tok/s
    Peak prefill
    11.2K tok/s
  • gemma-3n-E4B-it-FP8-dynamicH200 NVL 141GB · vllm 0.10.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    115.1 tok/s
    Peak prefill
    34.5K tok/s
  • Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    2,381.2 tok/s
    Peak prefill
    10.2K tok/s
  • Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    2,063.9 tok/s
    Peak prefill
    10.4K tok/s
  • Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    2,782.2 tok/s
    Peak prefill
    14.1K tok/s
  • Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    2,570.2 tok/s
    Peak prefill
    14.5K tok/s
  • Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    6,835.3 tok/s
    Peak prefill
    40.5K tok/s
  • Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    5,351.1 tok/s
    Peak prefill
    39K tok/s
  • Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    7,688.1 tok/s
    Peak prefill
    43.1K tok/s
  • Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    6,581.0 tok/s
    Peak prefill
    41.3K tok/s
  • Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    2,855.1 tok/s
    Peak prefill
    10.3K tok/s
  • Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    2,067.6 tok/s
    Peak prefill
    10.4K tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    3,315.5 tok/s
    Peak prefill
    14.3K tok/s
  • Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
    Peak decode
    2,574.4 tok/s
    Peak prefill
    14.6K tok/s
  • Qwen3.5-9BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
    Peak decode
    7,520.1 tok/s
    Peak prefill
    30.4K tok/s