शुरुआत करेंलीडरबोर्डDecode calculatorमॉडलOrganizationsReportsMatrix runsहार्डवेयरबेंचमार्कमार्केटप्लेसरेंटलProAPI दस्तावेज़
भाषा
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 14 of 28.
  • LFM2.5-1.2B-InstructH200 NVL 141GB · sglang 0.5.6.post3.dev9418+gc26ae0439 · BF165 contexts × 8 concurrency levels
    Peak decode
    5,352.3 tok/s
    Peak prefill
    112.8K tok/s
  • gemma-4-12B-it-int8-AutoRoundH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · INT85 contexts × 8 concurrency levels
    Peak decode
    2,146.1 tok/s
    Peak prefill
    7,255 tok/s
  • gemma-4-12B-it-int4-AutoRoundH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · INT46 contexts × 8 concurrency levels
    Peak decode
    2,337.2 tok/s
    Peak prefill
    8,267 tok/s
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    3,676.1 tok/s
    Peak prefill
    22.3K tok/s
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    4,507.0 tok/s
    Peak prefill
    24.4K tok/s
  • Inkling-SmallH200 SXM 141GB ×8 · vllm 0.29.0 · BF168 contexts × 8 concurrency levels
    Peak decode
    7,302.0 tok/s
    Peak prefill
    42.2K tok/s
  • Inkling-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    2,463.3 tok/s
    Peak prefill
    16.1K tok/s
  • MiMo-V2.5H200 SXM 141GB ×4 · vllm 0.29.0 · FP88 contexts × 8 concurrency levels
    Peak decode
    4,684.3 tok/s
    Peak prefill
    38K tok/s
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    4,509.9 tok/s
    Peak prefill
    24.4K tok/s
  • GLM-5.3-Flash-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · NVFP4+FP88 contexts × 8 concurrency levels
    Peak decode
    4,635.3 tok/s
    Peak prefill
    20.1K tok/s
  • GLM-5.3-Flash-BF16H200 SXM 141GB ×8 · vllm 0.28.1rc1.dev359+g98ed0856f · BF168 contexts × 8 concurrency levels
    Peak decode
    5,063.6 tok/s
    Peak prefill
    29.7K tok/s
  • Hy4-preview-FP8H200 SXM 141GB ×8 · vllm 0.29.0 · MXFP84 contexts × 8 concurrency levels
    Peak decode
    425.4 tok/s
    Peak prefill
    4,614 tok/s
  • Kimi-K2.7-CodeH200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,560.9 tok/s
    Peak prefill
    17.7K tok/s
  • Kimi-K2.6H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,576.9 tok/s
    Peak prefill
    17.6K tok/s
  • dots3-note-prev-fp8H200 SXM 141GB ×8 · vllm 0.29.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,180.1 tok/s
    Peak prefill
    21.1K tok/s
  • Trinity-Large-ThinkingH200 SXM 141GB ×8 · vllm 0.29.0 · BF167 contexts × 8 concurrency levels
    Peak decode
    4,631.0 tok/s
    Peak prefill
    46.8K tok/s
  • Trinity-Large-PreviewH200 SXM 141GB ×8 · vllm 0.29.0 · BF167 contexts × 8 concurrency levels
    Peak decode
    3,152.1 tok/s
    Peak prefill
    46.6K tok/s
  • GLM-5.3-FlashH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev10338+g787299797 · FP88 contexts × 8 concurrency levels
    Peak decode
    2,302.9 tok/s
    Peak prefill
    22.3K tok/s
  • Kimi-K2.6-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    630.8 tok/s
    Peak prefill
    18.7K tok/s
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    711.4 tok/s
    Peak prefill
    18.8K tok/s
  • Kimi-K2.5-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    603.0 tok/s
    Peak prefill
    17.8K tok/s
  • Kimi-K2.7-Code-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    620.1 tok/s
    Peak prefill
    18.7K tok/s
  • Kimi-K2.6H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    686.5 tok/s
    Peak prefill
    18.8K tok/s
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    340.4 tok/s
    Peak prefill
    18.8K tok/s
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    410.4 tok/s
    Peak prefill
    18.8K tok/s