Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 13 of 28.- Ling-3.0-tiny-baseH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 76.6K tok/s
- Ling-3.0-tiny-base-midtrainH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 78.4K tok/s
- Ling-3.0-flash-int4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · INT46 contexts × 8 concurrency levels
- Peak decode
- 3,139.0 tok/s
- Peak prefill
- 22.4K tok/s
- Ling-3.0-flash-fp8H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · FP86 contexts × 8 concurrency levels
- Peak decode
- 816.3 tok/s
- Peak prefill
- 33K tok/s
- Ling-3.0-flash-fp4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · MXFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,914.4 tok/s
- Peak prefill
- 15.8K tok/s
- Ling-3.0-flash-int4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,252.3 tok/s
- Peak prefill
- 22.2K tok/s
- Ling-2.6-flashH200 NVL 141GB ×2 · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF165 contexts × 8 concurrency levels
- Peak decode
- 2,891.5 tok/s
- Peak prefill
- 31.7K tok/s
- Ling-2.6-flash-int4H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · INT45 contexts × 8 concurrency levels
- Peak decode
- 2,202.7 tok/s
- Peak prefill
- 18.2K tok/s
- Ling-2.6-flash-fp8H200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · FP85 contexts × 8 concurrency levels
- Peak decode
- 1,696.9 tok/s
- Peak prefill
- 30.8K tok/s
- Ling-2.6-flash-baseH200 NVL 141GB · sglang 0.5.6.post3.dev8719+ge1a24a189 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,664.9 tok/s
- Peak prefill
- 30.9K tok/s
- Peak decode
- 1,809.1 tok/s
- Peak prefill
- 25.7K tok/s
- Peak decode
- 2,734.5 tok/s
- Peak prefill
- 47.5K tok/s
- Peak decode
- 504.1 tok/s
- Peak prefill
- 8,556 tok/s
- gemma-4-E4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · BF165 contexts × 8 concurrency levels
- Peak decode
- 5,060.8 tok/s
- Peak prefill
- 27.5K tok/s
- gemma-4-E2B-itH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · BF165 contexts × 8 concurrency levels
- Peak decode
- 11.3K tok/s
- Peak prefill
- 35.3K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,174.0 tok/s
- Peak prefill
- 18.4K tok/s
- gemma-4-12B-itH200 NVL 141GB · sglang 0.5.6.post3.dev5791+gf7d4e44d8 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,234.2 tok/s
- Peak prefill
- 11.8K tok/s
- Qwen3.8-27B-NVFP4-BF16-LMHeadH200 NVL 141GB · sglang 0.5.6.post3.dev9023+g73a255206 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,752.5 tok/s
- Peak prefill
- 6,480 tok/s
- Qwen3.8-Flash-Next-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9023+g73a255206 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,135.3 tok/s
- Peak prefill
- 27.4K tok/s
- Nanbeige4.2-3BH200 NVL 141GB · sglang 0.0.0.dev17517+g3e59d89e5 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,861.2 tok/s
- Peak prefill
- 27.9K tok/s
- LFM2.5-VL-3BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF163 contexts × 8 concurrency levels
- Peak decode
- 11.6K tok/s
- Peak prefill
- 65.3K tok/s
- LFM2.5-230MH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 20.5K tok/s
- Peak prefill
- 218.2K tok/s
- LFM2.5-2.6BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 9,578.7 tok/s
- Peak prefill
- 67.3K tok/s
- LFM2.5-2.6BH200 NVL 141GB · sglang 0.5.6.post3.dev9418+gc26ae0439 · BF165 contexts × 8 concurrency levels
- Peak decode
- 5,643.7 tok/s
- Peak prefill
- 65.6K tok/s
- LFM2.5-1.2B-InstructH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF165 contexts × 8 concurrency levels
- Peak decode
- 18.4K tok/s
- Peak prefill
- 114.6K tok/s