Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 22 of 30.- Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 5,019.0 tok/s
- Peak prefill
- 29.9K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 15.6K tok/s
- Peak prefill
- 113.3K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 17.1K tok/s
- Peak prefill
- 132.9K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 11.9K tok/s
- Peak prefill
- 118.1K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 17.1K tok/s
- Peak prefill
- 122.8K tok/s
- INTELLECT-3H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 514.0 tok/s
- Peak prefill
- 21.3K tok/s
- INTELLECT-3.1H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 516.6 tok/s
- Peak prefill
- 20.8K tok/s
- INTELLECT-3-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP85 contexts × 8 concurrency levels
- Peak decode
- 1,491.4 tok/s
- Peak prefill
- 24.2K tok/s
- INTELLECT-2H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 625.2 tok/s
- Peak prefill
- 8,458 tok/s
- Peak decode
- 3,232.3 tok/s
- Peak prefill
- 43.2K tok/s
- Peak decode
- 4,818.2 tok/s
- Peak prefill
- 32.2K tok/s
- Peak decode
- 5,181.7 tok/s
- Peak prefill
- 36.2K tok/s
- Peak decode
- 5,258.4 tok/s
- Peak prefill
- 41.2K tok/s
- Peak decode
- 3,178.6 tok/s
- Peak prefill
- 42.6K tok/s
- Peak decode
- 4,826.6 tok/s
- Peak prefill
- 33.6K tok/s
- Peak decode
- 5,199.9 tok/s
- Peak prefill
- 35.2K tok/s
- Peak decode
- 5,228.0 tok/s
- Peak prefill
- 42.4K tok/s
- Peak decode
- 1,466.6 tok/s
- Peak prefill
- 44K tok/s
- Peak decode
- 1,618.0 tok/s
- Peak prefill
- 33.2K tok/s
- Peak decode
- 1,695.8 tok/s
- Peak prefill
- 34.6K tok/s
- Peak decode
- 1,966.7 tok/s
- Peak prefill
- 42.6K tok/s
- Peak decode
- 3,800.8 tok/s
- Peak prefill
- 23.7K tok/s
- Peak decode
- 4,021.1 tok/s
- Peak prefill
- 24.2K tok/s
- Peak decode
- 3,595.4 tok/s
- Peak prefill
- 29.9K tok/s
- Peak decode
- 872.0 tok/s
- Peak prefill
- 8,929 tok/s