Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 4 of 30.- Kimi-K2.6-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 487.4 tok/s
- Peak prefill
- 2,929 tok/s
- Kimi-K2.7-CodeRTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · INT45 contexts × 8 concurrency levels
- Peak decode
- 374.8 tok/s
- Peak prefill
- 4,444 tok/s
- Kimi-K2.6RTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · INT45 contexts × 8 concurrency levels
- Peak decode
- 376.1 tok/s
- Peak prefill
- 4,514 tok/s
- Qwen3.5-397B-A17B-NVFP4-V2RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+g9fedd24fa · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,271.5 tok/s
- Peak prefill
- 11.8K tok/s
- Qwen3.5-397B-A17B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+g9fedd24fa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 2,214.5 tok/s
- Peak prefill
- 11.1K tok/s
- Qwen3.5-397B-A17B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,259.4 tok/s
- Peak prefill
- 5,452 tok/s
- Qwen3.5-397B-A17B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 939.0 tok/s
- Peak prefill
- 5,517 tok/s
- Qwen3.5-397B-A17B-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,942.3 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.5-397B-A17B-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,496.9 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.5-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,446.9 tok/s
- Peak prefill
- 9,958 tok/s
- Kimi-K2.7-Code-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.1.dev1+g9fedd24fa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 364.0 tok/s
- Peak prefill
- 6,206 tok/s
- Kimi-K2.7-CodeRTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
- Peak decode
- 283.8 tok/s
- Peak prefill
- 5,663 tok/s
- Kimi-K2.6RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
- Peak decode
- 285.4 tok/s
- Peak prefill
- 5,801 tok/s
- Qwen3.5-122B-A10B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 2,131.6 tok/s
- Peak prefill
- 9,953 tok/s
- Qwen3.5-122B-A10B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,477.9 tok/s
- Peak prefill
- 9,423 tok/s
- Qwen3.5-122B-A10B-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,050.4 tok/s
- Peak prefill
- 15.9K tok/s
- Qwen3.5-122B-A10B-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,193.0 tok/s
- Peak prefill
- 15.6K tok/s
- MiniMax-M2RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ge0317b28c · FP86 contexts × 8 concurrency levels
- Peak decode
- 630.5 tok/s
- Peak prefill
- 15.2K tok/s
- MiniMax-M2.1RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ge0317b28c · FP86 contexts × 8 concurrency levels
- Peak decode
- 612.7 tok/s
- Peak prefill
- 15.2K tok/s
- MiniMax-M2.5RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ge0317b28c · FP86 contexts × 8 concurrency levels
- Peak decode
- 922.3 tok/s
- Peak prefill
- 15.1K tok/s
- MiniMax-M2.7RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.1.dev1+ge0317b28c · FP86 contexts × 8 concurrency levels
- Peak decode
- 629.4 tok/s
- Peak prefill
- 15.3K tok/s
- Nex-N2-miniRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.1.dev1+ge0317b28c · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,420.1 tok/s
- Peak prefill
- 30.9K tok/s
- Step-3.7-Flash-FP8RTX PRO 6000 Blackwell Server Edition 96GB ×4 · sglang 0.0.0.dev1+gf92199b56 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,310.0 tok/s
- Peak prefill
- 15.5K tok/s
- Step-3.7-Flash-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×2 · sglang 0.0.0.dev1+gf92199b56 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 722.6 tok/s
- Peak prefill
- 15.9K tok/s
- Ling-3.0-tinyRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 4,912.1 tok/s
- Peak prefill
- 55.2K tok/s