Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 19 of 30.- Trinity-Large-Thinking-FP8-BlockH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · FP87 contexts × 8 concurrency levels
- Peak decode
- 3,864.0 tok/s
- Peak prefill
- 33.3K tok/s
- gemma-4-12B-itH200 NVL 141GB · sglang 0.0.0.dev16952+gd5bd07850 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,015.5 tok/s
- Peak prefill
- 10.1K tok/s
- Peak decode
- 4,637.7 tok/s
- Peak prefill
- 60.9K tok/s
- ZAYA1-8B-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,057.5 tok/s
- Peak prefill
- 55K tok/s
- ZAYA1-74B-preview-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,582.1 tok/s
- Peak prefill
- 20.4K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,521.1 tok/s
- Peak prefill
- 10.1K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,954.8 tok/s
- Peak prefill
- 41.8K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 887.0 tok/s
- Peak prefill
- 10.2K tok/s
- Peak decode
- 5,252.1 tok/s
- Peak prefill
- 30.5K tok/s
- Peak decode
- 6,429.2 tok/s
- Peak prefill
- 46.7K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,465.9 tok/s
- Peak prefill
- 42.5K tok/s
- Qwen3.5-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,214.8 tok/s
- Peak prefill
- 10.2K tok/s
- gpt-oss-20bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 5,651.7 tok/s
- Peak prefill
- 31.1K tok/s
- gpt-oss-120bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 2,383.3 tok/s
- Peak prefill
- 19K tok/s
- gemma-4-12B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,195.8 tok/s
- Peak prefill
- 19.5K tok/s
- gemma-4-31B-itH200 NVL 141GB ×2 · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,740.9 tok/s
- Peak prefill
- 14.4K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 10.1K tok/s
- Peak prefill
- 37.3K tok/s
- Step-3.7-Flash-NVFP4H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 3,572.1 tok/s
- Peak prefill
- 18.3K tok/s
- Step-3.7-Flash-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,443.8 tok/s
- Peak prefill
- 24.6K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,302.8 tok/s
- Peak prefill
- 38.6K tok/s
- Qwen3.6-35B-A3B-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 5,833.6 tok/s
- Peak prefill
- 31.1K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,506.0 tok/s
- Peak prefill
- 41.4K tok/s
- Qwen3.6-35B-A3B-FP8-dynamicH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,065.3 tok/s
- Peak prefill
- 41.4K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,554.0 tok/s
- Peak prefill
- 14.5K tok/s
- gemma-4-E4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 7,744.3 tok/s
- Peak prefill
- 40.6K tok/s