Erste SchritteLeaderboardDecode calculatorModelleOrganizationsReportsMatrix runsHardwareBenchmarksMarktplatzVermietungenProAPI-Doku
Sprache
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 27 of 28.
  • Qwen3.5-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,559.1 tok/s
    Peak prefill
    39.9K tok/s
  • Qwen3.5-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,344.8 tok/s
    Peak prefill
    40.4K tok/s
  • Qwen3.5-35B-A3B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,797.8 tok/s
    Peak prefill
    35.9K tok/s
  • Qwen3.5-35B-A3B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,351.2 tok/s
    Peak prefill
    36.5K tok/s
  • Qwen3.5-2BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.1K tok/s
    Peak prefill
    77.3K tok/s
  • Qwen3.5-2BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.3K tok/s
    Peak prefill
    80.6K tok/s
  • Qwen3.5-2B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,110.7 tok/s
    Peak prefill
    77.5K tok/s
  • Qwen3.5-2B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.4K tok/s
    Peak prefill
    79.8K tok/s
  • Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,843.2 tok/s
    Peak prefill
    9,631 tok/s
  • Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,183.1 tok/s
    Peak prefill
    9,731 tok/s
  • Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,910.6 tok/s
    Peak prefill
    6,160 tok/s
  • Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,223.8 tok/s
    Peak prefill
    6,236 tok/s
  • Qwen3.5-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,263.4 tok/s
    Peak prefill
    13.3K tok/s
  • Qwen3.5-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,582.8 tok/s
    Peak prefill
    13.7K tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    571.4 tok/s
    Peak prefill
    12.8K tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    2,006.6 tok/s
    Peak prefill
    13.2K tok/s
  • Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    4,794.2 tok/s
    Peak prefill
    28.4K tok/s
  • Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.3K tok/s
    Peak prefill
    99K tok/s
  • Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    16K tok/s
    Peak prefill
    98.7K tok/s
  • Qwen3.5-0.8B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,199.7 tok/s
    Peak prefill
    98.2K tok/s
  • Qwen3.5-0.8B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    15.7K tok/s
    Peak prefill
    104.3K tok/s
  • INTELLECT-3H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF165 contexts × 8 concurrency levels
    Peak decode
    607.4 tok/s
    Peak prefill
    16.3K tok/s
  • INTELLECT-3.1H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF165 contexts × 8 concurrency levels
    Peak decode
    604.2 tok/s
    Peak prefill
    16.4K tok/s
  • INTELLECT-2H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP164 contexts × 8 concurrency levels
    Peak decode
    481.2 tok/s
    Peak prefill
    7,964 tok/s
  • Laguna-XS.2H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,861.9 tok/s
    Peak prefill
    34.8K tok/s