Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 740
Newest first. Page 5 of 30.- Ling-3.0-tiny-int4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
- Peak decode
- 6,091.1 tok/s
- Peak prefill
- 57.5K tok/s
- Ling-3.0-tiny-fp8RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP85 contexts × 8 concurrency levels
- Peak decode
- 5,377.7 tok/s
- Peak prefill
- 60.5K tok/s
- Ling-3.0-tiny-baseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,471.3 tok/s
- Peak prefill
- 55.5K tok/s
- Ling-3.0-tiny-base-midtrainRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,563.8 tok/s
- Peak prefill
- 55.9K tok/s
- Ling-3.0-tinyRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF165 contexts × 8 concurrency levels
- Peak decode
- 6,109.2 tok/s
- Peak prefill
- 23.5K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.26.0 · NVFP43 contexts × 3 concurrency levels
- Peak decode
- 1,092.4 tok/s
- Peak prefill
- 6,659 tok/s
- Peak decode
- 80.6 tok/s
- Peak prefill
- 1,292 tok/s
- Ling-3.0-tiny-int4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · INT45 contexts × 8 concurrency levels
- Peak decode
- 5,636.2 tok/s
- Peak prefill
- 23.6K tok/s
- Ling-3.0-tiny-fp8RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · FP85 contexts × 8 concurrency levels
- Peak decode
- 6,226.5 tok/s
- Peak prefill
- 23.1K tok/s
- Ling-3.0-tiny-baseRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,051.4 tok/s
- Peak prefill
- 26.3K tok/s
- Ling-3.0-tiny-base-midtrainRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,054.6 tok/s
- Peak prefill
- 26.1K tok/s
- Peak decode
- 1,635.7 tok/s
- Peak prefill
- 30.8K tok/s
- Ling-3.0-flash-int4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · INT46 contexts × 8 concurrency levels
- Peak decode
- 572.1 tok/s
- Peak prefill
- 15.1K tok/s
- Peak decode
- 1,553.7 tok/s
- Peak prefill
- 11K tok/s
- Ling-3.0-flash-fp4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · MXFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 888.4 tok/s
- Peak prefill
- 19.9K tok/s
- Ling-2.6-flash-int4RTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · INT45 contexts × 8 concurrency levels
- Peak decode
- 988.4 tok/s
- Peak prefill
- 13K tok/s
- Muse-Glimmer-30BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 1,022.7 tok/s
- Peak prefill
- 6,383 tok/s
- Muse-Glimmer-30BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 759.7 tok/s
- Peak prefill
- 6,382 tok/s
- Muse-Glimmer-30BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF165 contexts × 8 concurrency levels
- Peak decode
- 766.2 tok/s
- Peak prefill
- 6,729 tok/s
- Muse-Glimmer-30BRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF165 contexts × 8 concurrency levels
- Peak decode
- 102.8 tok/s
- Peak prefill
- 6,028 tok/s
- Fara1.5-9BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,887.0 tok/s
- Peak prefill
- 21.5K tok/s
- Fara1.5-4BRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,786.2 tok/s
- Peak prefill
- 34.6K tok/s
- limite-1b-violettoRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 438.3 tok/s
- Peak prefill
- 63.7K tok/s
- limite-1b-baseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 6,830.6 tok/s
- Peak prefill
- 95.7K tok/s
- limite-1b-base-soupRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
- Peak decode
- 8,125.7 tok/s
- Peak prefill
- 95.4K tok/s