Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 13 of 30.- Qwen3.5-0.8B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
- Peak decode
- 10.8K tok/s
- Peak prefill
- 87.2K tok/s
- INTELLECT-2RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP164 contexts × 8 concurrency levels
- Peak decode
- 85.4 tok/s
- Peak prefill
- 5,276 tok/s
- LFM2.5-VL-3BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF163 contexts × 8 concurrency levels
- Peak decode
- 11.6K tok/s
- Peak prefill
- 47.1K tok/s
- LFM2.5-8B-A1BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 6,034.8 tok/s
- Peak prefill
- 42.2K tok/s
- LFM2.5-8B-A1BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9418+gc26ae0439 · BF165 contexts × 8 concurrency levels
- Peak decode
- 2,702.4 tok/s
- Peak prefill
- 42.2K tok/s
- LFM2.5-350MRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 2,956.5 tok/s
- Peak prefill
- 176.3K tok/s
- LFM2.5-230MRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak burst decode
- —
- Peak prefill
- 223.6K tok/s
- LFM2.5-2.6BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 10.5K tok/s
- Peak prefill
- 49.5K tok/s
- LFM2.5-2.6BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9418+gc26ae0439 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,759.1 tok/s
- Peak prefill
- 48.4K tok/s
- LFM2.5-1.2B-ThinkingRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 15.5K tok/s
- Peak prefill
- 63.4K tok/s
- LFM2.5-1.2B-InstructRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 15K tok/s
- Peak prefill
- 93.1K tok/s
- LFM2.5-1.2B-InstructRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev9418+gc26ae0439 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,655.7 tok/s
- Peak prefill
- 93.2K tok/s
- gemma-4-12B-it-int8-AutoRoundRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · INT85 contexts × 8 concurrency levels
- Peak decode
- 491.9 tok/s
- Peak prefill
- 8,163 tok/s
- Mistral-Small-4-119B-2603H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,139.4 tok/s
- Peak prefill
- 40.8K tok/s
- Ministral-3-14B-Base-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,864.8 tok/s
- Peak prefill
- 19.1K tok/s
- gemma-4-31B-it-int4-AutoRound-V2H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 56.4 tok/s
- Peak prefill
- 6,084 tok/s
- LFM2.5-350MH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 2,584.4 tok/s
- Peak prefill
- 189.8K tok/s
- North-Mini-Code-1.0H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF167 contexts × 8 concurrency levels
- Peak decode
- 4,032.8 tok/s
- Peak prefill
- 39.4K tok/s
- Qwen3.6-35B-A3B-NVFP4H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 5,952.1 tok/s
- Peak prefill
- 36.3K tok/s
- Qwen3.6-27B-NVFP4H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 913.1 tok/s
- Peak prefill
- 6,368 tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 4,544.9 tok/s
- Peak prefill
- 31K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,377.6 tok/s
- Peak prefill
- 45.6K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,619.5 tok/s
- Peak prefill
- 31.7K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,199.0 tok/s
- Peak prefill
- 29.2K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · sglang 0.0.0.dev17157+gccfcdd93a · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,297.0 tok/s
- Peak prefill
- 29.1K tok/s