Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 4 of 28.- limite-1b-violettoRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 438.3 tok/s
- Peak prefill
- 63.7K tok/s
- limite-1b-baseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 6,830.6 tok/s
- Peak prefill
- 95.7K tok/s
- limite-1b-base-soupRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 8,125.7 tok/s
- Peak prefill
- 95.4K tok/s
- ZAYA1-8BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.23.1rc1.dev772+gb1b99a08b · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,927.6 tok/s
- Peak prefill
- 52.3K tok/s
- ZAYA1-8B-legacyRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.21.1rc1.dev409+g2ce2f26c2 · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,307.7 tok/s
- Peak prefill
- 49.6K tok/s
- Qwen3.8-27BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev172+ge55909cad · BF166 contexts × 8 concurrency levels
- Peak decode
- 393.1 tok/s
- Peak prefill
- 6,516 tok/s
- Qwen3.6-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 511.3 tok/s
- Peak prefill
- 28.3K tok/s
- Qwen3.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,332.2 tok/s
- Peak prefill
- 20.7K tok/s
- Qwen3.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,777.1 tok/s
- Peak prefill
- 32.9K tok/s
- Qwen3.5-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 568.8 tok/s
- Peak prefill
- 28.8K tok/s
- Qwen3.5-27BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 450.1 tok/s
- Peak prefill
- 6,659 tok/s
- gpt-oss-20bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 1,982.5 tok/s
- Peak prefill
- 36.4K tok/s
- gemma-4-12B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 97.9 tok/s
- Peak prefill
- 10.7K tok/s
- Qwen3.8-27B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,348.3 tok/s
- Peak prefill
- 12.9K tok/s
- Qwen3.6-35B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 4,311.7 tok/s
- Peak prefill
- 40.4K tok/s
- Qwen3.6-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,177.6 tok/s
- Peak prefill
- 35.5K tok/s
- Qwen3.6-35B-A3B-FP8-dynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,705.9 tok/s
- Peak prefill
- 34.3K tok/s
- Muse-Glimmer-30BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,024.1 tok/s
- Peak prefill
- 6,382 tok/s
- Muse-Glimmer-30B-INT4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
- Peak decode
- 845.8 tok/s
- Peak prefill
- 5,771 tok/s
- Muse-Glimmer-30B-INT4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
- Peak decode
- 1,794.9 tok/s
- Peak prefill
- 5,773 tok/s
- Muse-Glimmer-30B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 1,435.5 tok/s
- Peak prefill
- 16.1K tok/s
- Muse-Glimmer-30B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 2,069.6 tok/s
- Peak prefill
- 16.1K tok/s
- Muse-Glimmer-30B-FP8-blockRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP85 contexts × 8 concurrency levels
- Peak decode
- 1,113.7 tok/s
- Peak prefill
- 9,036 tok/s
- Muse-Glimmer-30B-FP8-blockRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP85 contexts × 8 concurrency levels
- Peak decode
- 1,813.5 tok/s
- Peak prefill
- 9,063 tok/s
- gemma-4-E4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,899.2 tok/s
- Peak prefill
- 27.1K tok/s