Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 14 of 30.- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · sglang 0.0.0.dev17157+gccfcdd93a · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,024.8 tok/s
- Peak prefill
- 29.1K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF168 contexts × 8 concurrency levels
- Peak decode
- 4,501.9 tok/s
- Peak prefill
- 19.8K tok/s
- NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 1,053.8 tok/s
- Peak prefill
- 9,989 tok/s
- NVIDIA-Nemotron-3-Super-120B-A12B-BF16H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF168 contexts × 8 concurrency levels
- Peak decode
- 486.7 tok/s
- Peak prefill
- 10.5K tok/s
- DeepSeek-V4-Flash-nvfp4-DSparkH200 NVL 141GB ×2 · sglang 0.5.16 · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 757.8 tok/s
- Peak prefill
- 14.7K tok/s
- DeepSeek-V4-Flash-NVFP4H200 NVL 141GB ×2 · sglang 0.5.16 · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 266.0 tok/s
- Peak prefill
- 14.9K tok/s
- Shieldstral-1.0-3BH200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF163 contexts × 8 concurrency levels
- Peak decode
- 3,378.6 tok/s
- Peak prefill
- 51.7K tok/s
- Mistral-Medium-3.5-128BH200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 627.8 tok/s
- Peak prefill
- 5,218 tok/s
- Mistral-Medium-3.5-128BH200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 394.2 tok/s
- Peak prefill
- 5,063 tok/s
- Ministral-3-8B-Reasoning-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,313.9 tok/s
- Peak prefill
- 28.7K tok/s
- Ministral-3-8B-Instruct-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,523.4 tok/s
- Peak prefill
- 37.9K tok/s
- Ministral-3-3B-Reasoning-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,367.8 tok/s
- Peak prefill
- 50.1K tok/s
- Ministral-3-3B-Instruct-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,140.9 tok/s
- Peak prefill
- 62.1K tok/s
- Ministral-3-3B-Base-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,375.0 tok/s
- Peak prefill
- 47.5K tok/s
- Ministral-3-14B-Instruct-2512H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,031.7 tok/s
- Peak prefill
- 26.4K tok/s
- Muse-Glimmer-30BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 2,616.6 tok/s
- Peak prefill
- 10.4K tok/s
- Muse-Glimmer-30BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,661.9 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.8-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,101.1 tok/s
- Peak prefill
- 9,412 tok/s
- Muse-Glimmer-30BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,278.5 tok/s
- Peak prefill
- 10.3K tok/s
- Ling-3.0-tinyH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF165 contexts × 8 concurrency levels
- Peak decode
- 11.5K tok/s
- Peak prefill
- 78.2K tok/s
- Ling-3.0-tiny-int4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · INT45 contexts × 8 concurrency levels
- Peak decode
- 13.3K tok/s
- Peak prefill
- 73.6K tok/s
- Ling-3.0-tiny-fp8H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · FP85 contexts × 8 concurrency levels
- Peak decode
- 11.4K tok/s
- Peak prefill
- 90K tok/s
- Ling-3.0-tiny-baseH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 76.6K tok/s
- Ling-3.0-tiny-base-midtrainH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 78.4K tok/s
- Ling-3.0-flash-int4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · INT46 contexts × 8 concurrency levels
- Peak decode
- 3,139.0 tok/s
- Peak prefill
- 22.4K tok/s