Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 7 of 28.- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 5,609.6 tok/s
- Peak prefill
- 33.2K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF168 contexts × 8 concurrency levels
- Peak decode
- 2,924.1 tok/s
- Peak prefill
- 32.4K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 3,802.8 tok/s
- Peak prefill
- 44K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,072.5 tok/s
- Peak prefill
- 45.3K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,602.9 tok/s
- Peak prefill
- 32.1K tok/s
- Muse-Glimmer-30B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP85 contexts × 8 concurrency levels
- Peak decode
- 1,582.3 tok/s
- Peak prefill
- 6,382 tok/s
- Gemma-4-31B-IT-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 763.2 tok/s
- Peak prefill
- 8,242 tok/s
- Gemma-4-26B-A4B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 206.9 tok/s
- Peak prefill
- 34.8K tok/s
- Qwen3.6-35B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11207+g13e5dcf04 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 3,544.8 tok/s
- Peak prefill
- 33.9K tok/s
- Qwen3.6-27B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11207+g13e5dcf04 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,187.1 tok/s
- Peak prefill
- 6,940 tok/s
- Nanbeige4.2-3BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.26.1rc1.dev1235+g62f6de733 · BF166 contexts × 8 concurrency levels
- Peak decode
- 660.2 tok/s
- Peak prefill
- 18.3K tok/s
- Ministral-3-8B-Instruct-2512RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.31.1rc1.dev21+g678ef3358 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,918.2 tok/s
- Peak prefill
- 31.2K tok/s
- Ministral-3-14B-Reasoning-2512RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 678.1 tok/s
- Peak prefill
- 9,977 tok/s
- Ministral-3-14B-Instruct-2512RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,496.5 tok/s
- Peak prefill
- 16.1K tok/s
- Ministral-3-14B-Base-2512RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 682.1 tok/s
- Peak prefill
- 13.3K tok/s
- Shieldstral-1.0-3BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF163 contexts × 8 concurrency levels
- Peak decode
- 2,190.9 tok/s
- Peak prefill
- 39.4K tok/s
- Ministral-3-8B-Instruct-2512RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,789.1 tok/s
- Peak prefill
- 29.3K tok/s
- Ministral-3-14B-Instruct-2512RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · FP86 contexts × 8 concurrency levels
- Peak decode
- 937.3 tok/s
- Peak prefill
- 20.6K tok/s
- Ministral-3-14B-Base-2512RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF166 contexts × 8 concurrency levels
- Peak decode
- 649.4 tok/s
- Peak prefill
- 13.1K tok/s
- Devstral-Small-2-24B-Instruct-2512RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · FP86 contexts × 8 concurrency levels
- Peak decode
- 629.0 tok/s
- Peak prefill
- 11.8K tok/s
- Qwen3.6-27B-int4-AutoRoundRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 1,305.0 tok/s
- Peak prefill
- 6,104 tok/s
- Qwen3.6-27B-int4-AutoRoundRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 1,737.4 tok/s
- Peak prefill
- 6,167 tok/s
- Qwen3.8-27BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF166 contexts × 8 concurrency levels
- Peak decode
- 221.1 tok/s
- Peak prefill
- 6,672 tok/s
- granite-4.2-8bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,259.8 tok/s
- Peak prefill
- 17.8K tok/s
- granite-4.2-3bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,104.3 tok/s
- Peak prefill
- 36.9K tok/s