Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 693
Submitted by @keennay. Newest first. Page 6 of 28.- Qwen3.5-35B-A3B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,696.3 tok/s
- Peak prefill
- 36.7K tok/s
- Qwen3.5-2BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,242.0 tok/s
- Peak prefill
- 70.7K tok/s
- Qwen3.5-2BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,493.2 tok/s
- Peak prefill
- 75.2K tok/s
- Qwen3.5-2B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,426.7 tok/s
- Peak prefill
- 70.4K tok/s
- Qwen3.5-2B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,616.7 tok/s
- Peak prefill
- 74.5K tok/s
- Qwen3.5-0.8BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 10.7K tok/s
- Peak prefill
- 115.6K tok/s
- Qwen3.5-0.8B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,983.8 tok/s
- Peak prefill
- 108.6K tok/s
- Qwen3.5-0.8B-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 8,966.3 tok/s
- Peak prefill
- 116.9K tok/s
- Laguna-XS.2-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 5,312.5 tok/s
- Peak prefill
- 41.6K tok/s
- Laguna-XS.2-INT4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 5,189.9 tok/s
- Peak prefill
- 33.3K tok/s
- Laguna-XS.2-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,315.4 tok/s
- Peak prefill
- 37.1K tok/s
- Laguna-XS-2.1-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 2,565.2 tok/s
- Peak prefill
- 31.7K tok/s
- Laguna-XS-2.1-INT4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 1,251.6 tok/s
- Peak prefill
- 21.2K tok/s
- Laguna-XS-2.1-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,200.7 tok/s
- Peak prefill
- 31.9K tok/s
- Laguna-XS-2.1-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 1,265.7 tok/s
- Peak prefill
- 30.9K tok/s
- Laguna-XS-2.1-INT4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 1,021.4 tok/s
- Peak prefill
- 21.2K tok/s
- Laguna-XS-2.1-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,467.8 tok/s
- Peak prefill
- 32K tok/s
- gpt-oss-safeguard-20bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 7,397.8 tok/s
- Peak prefill
- 35.6K tok/s
- gpt-oss-safeguard-120bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 831.4 tok/s
- Peak prefill
- 23.6K tok/s
- gpt-oss-20bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 6,781.7 tok/s
- Peak prefill
- 36.3K tok/s
- gpt-oss-120bRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 838.8 tok/s
- Peak prefill
- 23.8K tok/s
- Qwen3.6-35B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 4,431.0 tok/s
- Peak prefill
- 35.7K tok/s
- Qwen3.6-27B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,154.7 tok/s
- Peak prefill
- 7,090 tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,567.4 tok/s
- Peak prefill
- 33.8K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,904.0 tok/s
- Peak prefill
- 33K tok/s