Erste SchritteLeaderboardDecode calculatorModelleOrganizationsReportsMatrix runsHardwareBenchmarksMarktplatzVermietungenProAPI-Doku
Sprache
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 740

Newest first. Page 16 of 30.
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    3,676.1 tok/s
    Peak prefill
    22.3K tok/s
    by @keennay
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    4,507.0 tok/s
    Peak prefill
    24.4K tok/s
    by @keennay
  • Inkling-SmallH200 SXM 141GB ×8 · vllm 0.29.0 · BF168 contexts × 8 concurrency levels
    Peak decode
    7,302.0 tok/s
    Peak prefill
    42.2K tok/s
    by @keennay
  • Inkling-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    2,463.3 tok/s
    Peak prefill
    16.1K tok/s
    by @keennay
  • MiMo-V2.5H200 SXM 141GB ×4 · vllm 0.29.0 · FP88 contexts × 8 concurrency levels
    Peak decode
    4,684.3 tok/s
    Peak prefill
    38K tok/s
    by @keennay
  • GLM-5.3-FlashH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · FP88 contexts × 8 concurrency levels
    Peak decode
    4,509.9 tok/s
    Peak prefill
    24.4K tok/s
    by @keennay
  • GLM-5.3-Flash-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev359+g98ed0856f · NVFP4+FP88 contexts × 8 concurrency levels
    Peak decode
    4,635.3 tok/s
    Peak prefill
    20.1K tok/s
    by @keennay
  • GLM-5.3-Flash-BF16H200 SXM 141GB ×8 · vllm 0.28.1rc1.dev359+g98ed0856f · BF168 contexts × 8 concurrency levels
    Peak decode
    5,063.6 tok/s
    Peak prefill
    29.7K tok/s
    by @keennay
  • Hy4-preview-FP8H200 SXM 141GB ×8 · vllm 0.29.0 · MXFP84 contexts × 8 concurrency levels
    Peak decode
    425.4 tok/s
    Peak prefill
    4,614 tok/s
    by @keennay
  • Kimi-K2.7-CodeH200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,560.9 tok/s
    Peak prefill
    17.7K tok/s
    by @keennay
  • Kimi-K2.6H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,576.9 tok/s
    Peak prefill
    17.6K tok/s
    by @keennay
  • dots3-note-prev-fp8H200 SXM 141GB ×8 · vllm 0.29.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,180.1 tok/s
    Peak prefill
    21.1K tok/s
    by @keennay
  • Trinity-Large-ThinkingH200 SXM 141GB ×8 · vllm 0.29.0 · BF167 contexts × 8 concurrency levels
    Peak decode
    4,631.0 tok/s
    Peak prefill
    46.8K tok/s
    by @keennay
  • Trinity-Large-PreviewH200 SXM 141GB ×8 · vllm 0.29.0 · BF167 contexts × 8 concurrency levels
    Peak decode
    3,152.1 tok/s
    Peak prefill
    46.6K tok/s
    by @keennay
  • GLM-5.3-FlashH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev10338+g787299797 · FP88 contexts × 8 concurrency levels
    Peak decode
    2,302.9 tok/s
    Peak prefill
    22.3K tok/s
    by @keennay
  • Kimi-K2.6-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    630.8 tok/s
    Peak prefill
    18.7K tok/s
    by @keennay
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    711.4 tok/s
    Peak prefill
    18.8K tok/s
    by @keennay
  • Kimi-K2.5-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    603.0 tok/s
    Peak prefill
    17.8K tok/s
    by @keennay
  • Kimi-K2.7-Code-NVFP4H200 SXM 141GB ×8 · vllm 0.29.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    620.1 tok/s
    Peak prefill
    18.7K tok/s
    by @keennay
  • Kimi-K2.6H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    686.5 tok/s
    Peak prefill
    18.8K tok/s
    by @keennay
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    340.4 tok/s
    Peak prefill
    18.8K tok/s
    by @keennay
  • Kimi-K2.5H200 SXM 141GB ×8 · vllm 0.29.0 · INT46 contexts × 8 concurrency levels
    Peak decode
    410.4 tok/s
    Peak prefill
    18.8K tok/s
    by @keennay
  • MiniMax-M2.5H200 SXM 141GB ×4 · vllm 0.29.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,962.4 tok/s
    Peak prefill
    32.9K tok/s
    by @keennay
  • MiniMax-M2.7H200 SXM 141GB ×4 · vllm 0.29.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,264.3 tok/s
    Peak prefill
    35.1K tok/s
    by @keennay
  • MiniMax-M3-MXFP8H200 SXM 141GB ×8 · vllm 0.29.0 · MXFP88 contexts × 8 concurrency levels
    Peak decode
    629.0 tok/s
    Peak prefill
    18.8K tok/s
    by @keennay