Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 9 of 28.- Olmo-3.1-32B-Instruct-DPORTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev151+g73a78e6f1 · BF164 contexts × 8 concurrency levels
- Peak decode
- 146.6 tok/s
- Peak prefill
- 5,594 tok/s
- Olmo-3-7B-ThinkRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF164 contexts × 8 concurrency levels
- Peak decode
- 686.0 tok/s
- Peak prefill
- 22.9K tok/s
- Olmo-3-7B-InstructRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev151+g73a78e6f1 · BF164 contexts × 8 concurrency levels
- Peak decode
- 685.0 tok/s
- Peak prefill
- 23K tok/s
- Olmo-3-32B-ThinkRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF164 contexts × 8 concurrency levels
- Peak decode
- 146.8 tok/s
- Peak prefill
- 5,582 tok/s
- GLM-5.3-FlashRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev452+g73a583112 · FP88 contexts × 8 concurrency levels
- Peak decode
- 2,462.6 tok/s
- Peak prefill
- 11.2K tok/s
- GLM-5.3-FlashRTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11345+g028ac64f7 · FP88 contexts × 8 concurrency levels
- Peak decode
- 1,356.3 tok/s
- Peak prefill
- 10.7K tok/s
- GLM-5.3-FlashRTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11345+g028ac64f7 · FP88 contexts × 8 concurrency levels
- Peak decode
- 3,095.7 tok/s
- Peak prefill
- 11.8K tok/s
- GLM-5.3-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 466.6 tok/s
- Peak prefill
- 4,237 tok/s
- GLM-5.3-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev167+g417b0b6aa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 452.6 tok/s
- Peak prefill
- 4,171 tok/s
- GLM-5.3-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 501.8 tok/s
- Peak prefill
- 4,282 tok/s
- GLM-5.3-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev167+g417b0b6aa · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 486.3 tok/s
- Peak prefill
- 4,052 tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+g50a061f79 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,501.2 tok/s
- Peak prefill
- 14.4K tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+g50a061f79 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,120.6 tok/s
- Peak prefill
- 14.8K tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,966.6 tok/s
- Peak prefill
- 15.2K tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,535.5 tok/s
- Peak prefill
- 15.9K tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,956.0 tok/s
- Peak prefill
- 15.3K tok/s
- Qwen3.8-Flash-Next-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,552.3 tok/s
- Peak prefill
- 16K tok/s
- GLM-5.3-Flash-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev162+g2617fe938 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 1,121.8 tok/s
- Peak prefill
- 12.5K tok/s
- GLM-5.3-FlashRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev162+g2617fe938 · FP88 contexts × 8 concurrency levels
- Peak decode
- 2,455.9 tok/s
- Peak prefill
- 11.1K tok/s
- GLM-5.3-FlashRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev162+g2617fe938 · FP88 contexts × 8 concurrency levels
- Peak decode
- 1,966.4 tok/s
- Peak prefill
- 10.7K tok/s
- MiMo-V2.6-Pro-RLRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.1.dev1+ged073f5eb · MXFP4+FP87 contexts × 8 concurrency levels
- Peak decode
- 553.8 tok/s
- Peak prefill
- 5,802 tok/s
- MiMo-V2.6-Flash-RLRTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ged073f5eb · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,567.8 tok/s
- Peak prefill
- 12.7K tok/s
- MiMo-V2.6-Flash-RLRTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ged073f5eb · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,527.4 tok/s
- Peak prefill
- 12.7K tok/s
- Qwen3.8-Flash-NextRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,894.0 tok/s
- Peak prefill
- 14.1K tok/s
- Qwen3.8-Flash-NextRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.1.dev1+g50a061f79 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,349.5 tok/s
- Peak prefill
- 12.5K tok/s