Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 18 of 30.- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,980.0 tok/s
- Peak prefill
- 20.1K tok/s
- Peak decode
- 1,335.6 tok/s
- Peak prefill
- 29.8K tok/s
- Peak decode
- 3,200.8 tok/s
- Peak prefill
- 18.2K tok/s
- Qwen3.8-Flash-Next-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 5,893.3 tok/s
- Peak prefill
- 28.5K tok/s
- Qwen3.8-27B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 6,984.1 tok/s
- Peak prefill
- 26.5K tok/s
- Qwen3.5-397B-A17B-NVFP4-V2H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,363.0 tok/s
- Peak prefill
- 15K tok/s
- Qwen3.5-397B-A17B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 2,249.8 tok/s
- Peak prefill
- 14.6K tok/s
- NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,329.6 tok/s
- Peak prefill
- 10.1K tok/s
- Peak decode
- 11.5K tok/s
- Peak prefill
- 63.1K tok/s
- Peak decode
- 11.3K tok/s
- Peak prefill
- 63K tok/s
- MiniMax-M2.7H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,914.9 tok/s
- Peak prefill
- 31.9K tok/s
- Peak decode
- 2,726.2 tok/s
- Peak prefill
- 34K tok/s
- Peak decode
- 3,640.4 tok/s
- Peak prefill
- 28.1K tok/s
- Peak decode
- 2,668.4 tok/s
- Peak prefill
- 33.7K tok/s
- MiniMax-M2.1H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,829.6 tok/s
- Peak prefill
- 31.5K tok/s
- Peak decode
- 2,698.3 tok/s
- Peak prefill
- 33.8K tok/s
- MiniMax-M2H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,907.7 tok/s
- Peak prefill
- 31.4K tok/s
- Peak decode
- 2,744.7 tok/s
- Peak prefill
- 33.5K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev317+g1356635d8 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,553.4 tok/s
- Peak prefill
- 21.6K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev317+g1356635d8 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,158.3 tok/s
- Peak prefill
- 21.8K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev9812+gd5a3b7725 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 136.9 tok/s
- Peak prefill
- 21.2K tok/s
- Peak decode
- 2,793.5 tok/s
- Peak prefill
- 22.8K tok/s
- Peak decode
- 8,783.5 tok/s
- Peak prefill
- 41.6K tok/s
- Ling-3.0-flashH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,881.5 tok/s
- Peak prefill
- 53.7K tok/s
- command-a-plus-05-2026-bf16H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,354.6 tok/s
- Peak prefill
- 24.6K tok/s