Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 17 of 28.- Peak decode
- 8,783.5 tok/s
- Peak prefill
- 41.6K tok/s
- Ling-3.0-flashH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,881.5 tok/s
- Peak prefill
- 53.7K tok/s
- command-a-plus-05-2026-bf16H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,354.6 tok/s
- Peak prefill
- 24.6K tok/s
- Trinity-Large-Thinking-FP8-BlockH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · FP87 contexts × 8 concurrency levels
- Peak decode
- 3,864.0 tok/s
- Peak prefill
- 33.3K tok/s
- gemma-4-12B-itH200 NVL 141GB · sglang 0.0.0.dev16952+gd5bd07850 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,015.5 tok/s
- Peak prefill
- 10.1K tok/s
- Peak decode
- 4,637.7 tok/s
- Peak prefill
- 60.9K tok/s
- ZAYA1-8B-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,057.5 tok/s
- Peak prefill
- 55K tok/s
- ZAYA1-74B-preview-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,582.1 tok/s
- Peak prefill
- 20.4K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,521.1 tok/s
- Peak prefill
- 10.1K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,954.8 tok/s
- Peak prefill
- 41.8K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 887.0 tok/s
- Peak prefill
- 10.2K tok/s
- Peak decode
- 5,252.1 tok/s
- Peak prefill
- 30.5K tok/s
- Peak decode
- 6,429.2 tok/s
- Peak prefill
- 46.7K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,465.9 tok/s
- Peak prefill
- 42.5K tok/s
- Qwen3.5-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,214.8 tok/s
- Peak prefill
- 10.2K tok/s
- gpt-oss-20bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 5,651.7 tok/s
- Peak prefill
- 31.1K tok/s
- gpt-oss-120bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 2,383.3 tok/s
- Peak prefill
- 19K tok/s
- gemma-4-12B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,195.8 tok/s
- Peak prefill
- 19.5K tok/s
- gemma-4-31B-itH200 NVL 141GB ×2 · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,740.9 tok/s
- Peak prefill
- 14.4K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 10.1K tok/s
- Peak prefill
- 37.3K tok/s
- Step-3.7-Flash-NVFP4H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 3,572.1 tok/s
- Peak prefill
- 18.3K tok/s
- Step-3.7-Flash-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,443.8 tok/s
- Peak prefill
- 24.6K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,302.8 tok/s
- Peak prefill
- 38.6K tok/s
- Qwen3.6-35B-A3B-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 5,833.6 tok/s
- Peak prefill
- 31.1K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,506.0 tok/s
- Peak prefill
- 41.4K tok/s