Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 20 of 30.- gemma-4-E2B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 13.1K tok/s
- Peak prefill
- 62K tok/s
- gemma-4-31B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 616.9 tok/s
- Peak prefill
- 7,892 tok/s
- gemma-4-31B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 1,143.4 tok/s
- Peak prefill
- 4,486 tok/s
- gemma-4-31B-it-FP8-blockH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,189.1 tok/s
- Peak prefill
- 11.4K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,389.3 tok/s
- Peak prefill
- 40.2K tok/s
- gemma-4-26B-A4B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 4,355.9 tok/s
- Peak prefill
- 24.7K tok/s
- gemma-4-12B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,491.4 tok/s
- Peak prefill
- 19.4K tok/s
- gemma-4-12B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 3,875.8 tok/s
- Peak prefill
- 11.2K tok/s
- Peak decode
- 115.1 tok/s
- Peak prefill
- 34.5K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,381.2 tok/s
- Peak prefill
- 10.2K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,063.9 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,782.2 tok/s
- Peak prefill
- 14.1K tok/s
- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,570.2 tok/s
- Peak prefill
- 14.5K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,835.3 tok/s
- Peak prefill
- 40.5K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,351.1 tok/s
- Peak prefill
- 39K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 7,688.1 tok/s
- Peak prefill
- 43.1K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,581.0 tok/s
- Peak prefill
- 41.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,855.1 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,067.6 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,315.5 tok/s
- Peak prefill
- 14.3K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,574.4 tok/s
- Peak prefill
- 14.6K tok/s
- Peak decode
- 7,520.1 tok/s
- Peak prefill
- 30.4K tok/s
- Peak decode
- 6,571.6 tok/s
- Peak prefill
- 31.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,561.1 tok/s
- Peak prefill
- 29.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,570.2 tok/s
- Peak prefill
- 32K tok/s