Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 7 of 30.- gemma-4-31B-it-FP8-blockRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 360.4 tok/s
- Peak prefill
- 6,106 tok/s
- gemma-4-26B-A4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,064.5 tok/s
- Peak prefill
- 21.5K tok/s
- gemma-4-26B-A4B-it-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 176.9 tok/s
- Peak prefill
- 31.5K tok/s
- gemma-4-26B-A4B-it-FP8-dynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 384.8 tok/s
- Peak prefill
- 34.1K tok/s
- gemma-4-12B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,570.4 tok/s
- Peak prefill
- 11K tok/s
- gemma-4-12B-it-FP8-DynamicRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 714.8 tok/s
- Peak prefill
- 19K tok/s
- Qwen3.8-27B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 885.9 tok/s
- Peak prefill
- 12.8K tok/s
- Qwen3.8-27B-NVFP4-BF16-LMHeadRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,196.1 tok/s
- Peak prefill
- 13.4K tok/s
- Qwen3.8-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.6.post3.dev11565+gfa7784d14 · FP86 contexts × 8 concurrency levels
- Peak decode
- 515.0 tok/s
- Peak prefill
- 9,770 tok/s
- Qwen3.6-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,698.2 tok/s
- Peak prefill
- 37.1K tok/s
- Qwen3.6-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,746.3 tok/s
- Peak prefill
- 37.6K tok/s
- Qwen3.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,971.1 tok/s
- Peak prefill
- 20.5K tok/s
- Qwen3.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,431.9 tok/s
- Peak prefill
- 21.3K tok/s
- Qwen3.5-9B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,683.4 tok/s
- Peak prefill
- 20.4K tok/s
- Qwen3.5-9B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,422.0 tok/s
- Peak prefill
- 21.4K tok/s
- Qwen3.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,224.8 tok/s
- Peak prefill
- 33K tok/s
- Qwen3.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,354.8 tok/s
- Peak prefill
- 33.9K tok/s
- Qwen3.5-4B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,307.7 tok/s
- Peak prefill
- 32.9K tok/s
- Qwen3.5-4B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,403.3 tok/s
- Peak prefill
- 34.5K tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 2,739.2 tok/s
- Peak prefill
- 13.6K tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,682.0 tok/s
- Peak prefill
- 13.2K tok/s
- Qwen3.5-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,907.2 tok/s
- Peak prefill
- 37.1K tok/s
- Qwen3.5-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,696.3 tok/s
- Peak prefill
- 36.7K tok/s
- Qwen3.5-2BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,242.0 tok/s
- Peak prefill
- 70.7K tok/s
- Qwen3.5-2BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,493.2 tok/s
- Peak prefill
- 75.2K tok/s