Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 3 of 30.- Qwen3.8-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,322.3 tok/s
- Peak prefill
- 8,440 tok/s
- Qwen3.8-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,447.8 tok/s
- Peak prefill
- 8,591 tok/s
- Qwen3.6-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,960.1 tok/s
- Peak prefill
- 32K tok/s
- Qwen3.6-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,551.0 tok/s
- Peak prefill
- 31.4K tok/s
- Qwen3.6-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,108.1 tok/s
- Peak prefill
- 8,398 tok/s
- Qwen3.6-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,450.7 tok/s
- Peak prefill
- 8,599 tok/s
- Qwen3.5-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,972.8 tok/s
- Peak prefill
- 32K tok/s
- Qwen3.5-35B-A3BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,517.0 tok/s
- Peak prefill
- 31.4K tok/s
- Qwen3.5-35B-A3B-BaseRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,367.7 tok/s
- Peak prefill
- 31.9K tok/s
- Qwen3.5-35B-A3B-BaseRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,240.3 tok/s
- Peak prefill
- 32K tok/s
- Qwen3.5-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,208.6 tok/s
- Peak prefill
- 8,403 tok/s
- Qwen3.5-27BRTX PRO 6000 Blackwell Server Edition 96GB ×2 · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,451.6 tok/s
- Peak prefill
- 8,620 tok/s
- Qwen3.6-35B-A3B-TT-Mixed-BFP4-BFP8-P150Tenstorrent P150 32GB · vllm 0.1.dev14187+g03fa3af2e · BFP46 contexts × 5 concurrency levels
- Peak decode
- 628.5 tok/s
- Peak prefill
- 2,014 tok/s
- Qwen3.6-35B-A3B-TT-Mixed-BFP4-BFP8-P150Tenstorrent P150 32GB · vllm 0.1.dev14187+g03fa3af2e · BFP46 contexts × 1 concurrency level
- Peak decode
- 130.4 tok/s
- Peak prefill
- 2,055 tok/s
- Qwen3.6-35B-A3B-TT-Mixed-BFP4-BFP8-P150Tenstorrent P150 32GB · vllm 0.1.dev14187+g03fa3af2e · BFP44 contexts × 5 concurrency levels
- Peak decode
- 626.2 tok/s
- Peak prefill
- 2,016 tok/s
- LFM2.5-24B-A2BTenstorrent P150 32GB · vllm 0.1.dev14187+g03fa3af2e · BFP46 contexts × 5 concurrency levels
- Peak decode
- 805.5 tok/s
- Peak prefill
- 10.6K tok/s
- Kimi-K2.6-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.1.dev1+g9fedd24fa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 362.0 tok/s
- Peak prefill
- 6,216 tok/s
- Qwen3.8-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,150.0 tok/s
- Peak prefill
- 9,767 tok/s
- Qwen3.8-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,449.2 tok/s
- Peak prefill
- 9,925 tok/s
- Qwen3.6-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,424.6 tok/s
- Peak prefill
- 9,831 tok/s
- Qwen3.6-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,449.5 tok/s
- Peak prefill
- 9,942 tok/s
- Qwen3.5-27B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,297.9 tok/s
- Peak prefill
- 6,145 tok/s
- Qwen3.5-27B-GPTQ-Int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,447.2 tok/s
- Peak prefill
- 6,241 tok/s
- Qwen3.5-27B-FP8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,426.1 tok/s
- Peak prefill
- 9,849 tok/s
- Kimi-K2.7-Code-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB ×8 · sglang 0.5.6.post3.dev11412+g964c45cf3 · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 489.0 tok/s
- Peak prefill
- 2,989 tok/s