Speed test matrix
MiniMax-M2.1 on RTX PRO 6000 Blackwell Server Edition 96GB ×4
vllm · 0.1.dev1+ge0317b28c · FP86 contexts (8K–187K) × 8 concurrency levels (1–128) · prefill at 8K, 16K, 32K, 64K, 128K, 187K, 192K
Hardware
- Type
- Discrete GPU
- GPU
- RTX PRO 6000 Blackwell Server Edition
- GPU count
- 4
- VRAM
- 384 GB (4 × 96 GB)
- OS
- linux
Engine
- Engine
- vllm
- Version
- 0.1.dev1+ge0317b28c
- Quantization
- FP8
- Backend
- cuda
- Repository
- github.com/vllm-project/vllm
- Build
- cuda-13.2
- Commit
e0317b28ce3b
Run
- Served model
- minimax-m2.1
- Revision
- main
- Benchmark tool
- llm-inference-bench v0.7.9
- Decode test
- Fixed duration
- Prefill mode
- Measured during decode warm-up
- Time per measurement
- 30 s
- Max output
- 4,096 tokens
Launch command
"$HOME/env_rtxpro6k-minimaxai-vllm-e0317b28ce3b/bin/vllm" serve MiniMaxAI/MiniMax-M2.1 --served-model-name minimax-m2.1 --trust-remote-code --tensor-parallel-size 4 --reasoning-parser minimax_m2_append_think --enable-auto-tool-choice --tool-call-parser minimax_m2 --max-model-len 196608 --gpu-memory-utilization 0.82 --host 0.0.0.0 --port 8000 --api-key <redacted> --attention-backend FLASH_ATTN --linear-backend triton --moe-backend triton
Prefill
prompt tok/s by context length8K context15.2K539 ms TTFT
16K context14.3K1,137 ms TTFT
32K context12.5K2,582 ms TTFT
64K context9,8756,517 ms TTFT
128K context6,84718,775 ms TTFT
187K context5,34635,116 ms TTFT
192K context5,17737,223 ms TTFT
Sustained decode
total tok/s · rows are context length, columns are simultaneous requests| Context | 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 |
|---|---|---|---|---|---|---|---|---|
| 8K | 85.949 ms TTFT | 154.975 ms TTFT | Looped | 391.0116 ms TTFT | 612.7196 ms TTFT | Over capacity | Over capacity | Over capacity |
| 16K | 82.069 ms TTFT | Looped | 252.7124 ms TTFT | 370.4184 ms TTFT | 562.0280 ms TTFT | Over capacity | Over capacity | Over capacity |
| 32K | 77.7120 ms TTFT | 138.0162 ms TTFT | 231.2210 ms TTFT | 332.0298 ms TTFT | Over capacity | Over capacity | Over capacity | Over capacity |
| 64K | Looped | 121.5283 ms TTFT | Looped | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity |
| 128K | 55.2427 ms TTFT | 97.6569 ms TTFT | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity |
| 187K | 47.7659 ms TTFT | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity | Over capacity |
Looped: the model kept repeating the same text, so the benchmark discarded this measurement. Over capacity: not run because this many requests at this context would exceed the server's KV cache.
TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.
All measurements
Latencies are medians. Total tok/s is output across all simultaneous requests.| Phase | Context | Requests at once | Status | Total tok/s | Per request tok/s | TTFT | Time between tokens | Average active requests | Completed requests | Output tokens | Measured for |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Prefill | 8K | 1 | Valid | 15.2K | — | 539 ms | — | — | 1 | — | — |
| Prefill | 16K | 1 | Valid | 14.3K | — | 1,137 ms | — | — | 1 | — | — |
| Prefill | 32K | 1 | Valid | 12.5K | — | 2,582 ms | — | — | 1 | — | — |
| Prefill | 64K | 1 | Valid | 9,875.0 | — | 6,517 ms | — | — | 1 | — | — |
| Prefill | 128K | 1 | Valid | 6,847.0 | — | 18,775 ms | — | — | 1 | — | — |
| Prefill | 187K | 1 | Valid | 5,346.0 | — | 35,116 ms | — | — | 1 | — | — |
| Prefill | 192K | 1 | Valid | 5,177.0 | — | 37,223 ms | — | — | 1 | — | — |
| Sustained decode | 8K | 1 | Valid | 85.9 | 86.0 | 49 ms | 11.6 ms | 1.0 | 0 | 2,576 | 30.0 s |
| Sustained decode | 8K | 2 | Valid | 154.9 | 77.5 | 75 ms | 12.9 ms | 2.0 | 0 | 4,646 | 30.0 s |
| Sustained decode | 8K | 4 | Looped | — | — | — | — | — | 0 | 0 | — |
| Sustained decode | 8K | 8 | Valid | 391.0 | 49.1 | 116 ms | 20.4 ms | 8.0 | 0 | 11,728 | 30.0 s |
| Sustained decode | 8K | 16 | Valid | 612.7 | 38.7 | 196 ms | 25.9 ms | 16.0 | 0 | 18,368 | 30.0 s |
| Sustained decode | 8K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 8K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 8K | 128 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 16K | 1 | Valid | 82.0 | 82.0 | 69 ms | 12.2 ms | 1.0 | 0 | 2,459 | 30.0 s |
| Sustained decode | 16K | 2 | Looped | — | — | — | — | — | 0 | 0 | — |
| Sustained decode | 16K | 4 | Valid | 252.7 | 63.2 | 124 ms | 15.8 ms | 4.0 | 0 | 7,580 | 30.0 s |
| Sustained decode | 16K | 8 | Valid | 370.4 | 46.4 | 184 ms | 21.5 ms | 8.0 | 0 | 11,104 | 30.0 s |
| Sustained decode | 16K | 16 | Valid | 562.0 | 35.4 | 280 ms | 28.3 ms | 16.0 | 0 | 16,848 | 30.0 s |
| Sustained decode | 16K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 16K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 16K | 128 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 32K | 1 | Valid | 77.7 | 77.7 | 120 ms | 12.9 ms | 1.0 | 0 | 2,330 | 30.0 s |
| Sustained decode | 32K | 2 | Valid | 138.0 | 69.1 | 162 ms | 14.5 ms | 2.0 | 0 | 4,140 | 30.0 s |
| Sustained decode | 32K | 4 | Valid | 231.2 | 57.9 | 210 ms | 17.3 ms | 4.0 | 0 | 6,936 | 30.0 s |
| Sustained decode | 32K | 8 | Valid | 332.0 | 41.6 | 298 ms | 24.0 ms | 8.0 | 0 | 9,952 | 30.0 s |
| Sustained decode | 32K | 16 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 32K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 32K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 32K | 128 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 64K | 1 | Looped | — | — | — | — | — | 0 | 0 | — |
| Sustained decode | 64K | 2 | Valid | 121.5 | 60.8 | 283 ms | 16.5 ms | 2.0 | 0 | 3,644 | 30.0 s |
| Sustained decode | 64K | 4 | Looped | — | — | — | — | — | 0 | 0 | — |
| Sustained decode | 64K | 8 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 64K | 16 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 64K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 64K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 64K | 128 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 1 | Valid | 55.2 | 55.2 | 427 ms | 18.1 ms | 1.0 | 0 | 1,655 | 30.0 s |
| Sustained decode | 128K | 2 | Valid | 97.6 | 48.8 | 569 ms | 20.5 ms | 2.0 | 0 | 2,928 | 30.0 s |
| Sustained decode | 128K | 4 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 8 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 16 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 128K | 128 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 1 | Valid | 47.7 | 47.8 | 659 ms | 20.9 ms | 1.0 | 0 | 1,432 | 30.0 s |
| Sustained decode | 187K | 2 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 4 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 8 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 16 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 32 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 64 | Over capacity | — | — | — | — | — | — | — | — |
| Sustained decode | 187K | 128 | Over capacity | — | — | — | — | — | — | — | — |
Looped: the model kept repeating the same text, so the benchmark discarded this measurement. Over capacity: not run because this many requests at this context would exceed the server's KV cache.