Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 17 of 30.- Peak decode
- 181.2 tok/s
- Peak prefill
- 26.9K tok/s
- Peak decode
- 4,673.1 tok/s
- Peak prefill
- 35K tok/s
- Peak decode
- 4,829.4 tok/s
- Peak prefill
- 35.5K tok/s
- Peak decode
- 366.5 tok/s
- Peak prefill
- 18.7K tok/s
- Peak decode
- 398.9 tok/s
- Peak prefill
- 18.7K tok/s
- GLM-5.3-Flash-NVFP4H200 SXM 141GB ×4 · sglang 0.5.6.post3.dev10338+g787299797 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 4,782.9 tok/s
- Peak prefill
- 17.6K tok/s
- Peak decode
- 1,798.9 tok/s
- Peak prefill
- 14.3K tok/s
- GLM-5.3-NVFP4H200 SXM 141GB ×8 · sglang 0.5.6.post3.dev10338+g787299797 · NVFP47 contexts × 8 concurrency levels
- Peak decode
- 1,513.7 tok/s
- Peak prefill
- 9,689 tok/s
- Peak decode
- 1,561.6 tok/s
- Peak prefill
- 10.2K tok/s
- Peak decode
- 1,459.2 tok/s
- Peak prefill
- 8,942 tok/s
- GLM-5.3-NVFP4H200 SXM 141GB ×8 · sglang 0.5.6.post3.dev10338+g787299797 · NVFP47 contexts × 8 concurrency levels
- Peak decode
- 1,466.0 tok/s
- Peak prefill
- 9,696 tok/s
- GLM-5.3-NVFP4H200 SXM 141GB ×8 · sglang 0.5.6.post3.dev10338+g787299797 · NVFP47 contexts × 8 concurrency levels
- Peak decode
- 1,471.1 tok/s
- Peak prefill
- 9,724 tok/s
- GLM-5.3-Flash-NVFP4H200 SXM 141GB ×4 · sglang 0.5.6.post3.dev10338+g787299797 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 2,480.0 tok/s
- Peak prefill
- 16.5K tok/s
- Peak decode
- 1,056.5 tok/s
- Peak prefill
- 11.9K tok/s
- Peak decode
- 1,203.0 tok/s
- Peak prefill
- 11.9K tok/s
- Peak decode
- 914.6 tok/s
- Peak prefill
- 12.1K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,950.5 tok/s
- Peak prefill
- 35.5K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.dev1+g50a061f79 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,358.8 tok/s
- Peak prefill
- 30.3K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,931.5 tok/s
- Peak prefill
- 34.8K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,547.2 tok/s
- Peak prefill
- 36.6K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.dev1+g50a061f79 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,373.4 tok/s
- Peak prefill
- 31K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,511.0 tok/s
- Peak prefill
- 36.7K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,258.2 tok/s
- Peak prefill
- 16.2K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 5,277.2 tok/s
- Peak prefill
- 20.6K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 3,016.1 tok/s
- Peak prefill
- 20.1K tok/s