Speed test matrix
Qwen3.8-Flash-Next-W3A8-Paiton-RDNA4 on Radeon AI Pro R9700 32GB ×2
vllm · 0.29.0 · W3A81 context (0) × 2 concurrency levels (1–8) · prefill at 8K, 64K
Hardware
- Type
- Discrete GPU
- GPU
- Radeon AI Pro R9700
- GPU count
- 2
- VRAM
- 64 GB (2 × 32 GB)
- CPU
- AMD Ryzen Threadripper 3960X 24-Core Processor
- RAM
- 251 GB
- OS
- Linux
Engine
- Engine
- vllm
- Version
- 0.29.0
- Quantization
- W3A8
Run
- Served model
- Qwen3.8-Flash-Next
- Revision
- main
- Benchmark tool
- llm-inference-bench v0.7.7
- Decode test
- Fixed duration
- Prefill mode
- Measured during decode warm-up
- Time per measurement
- 30 s
- Max output
- 8,192 tokens
Prefill
prompt tok/s by context length8K context8,292988 ms TTFT
64K context8,2057,813 ms TTFT
Sustained decode
total tok/s · rows are context length, columns are simultaneous requests| Context | 1 | 8 |
|---|---|---|
| 0 | 226.560 ms TTFT | 745.2198 ms TTFT |
TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.
All measurements
Latencies are medians. Total tok/s is output across all simultaneous requests.| Phase | Context | Requests at once | Status | Total tok/s | Per request tok/s | TTFT | Time between tokens | Average active requests | Completed requests | Output tokens | Measured for |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Prefill | 8K | 1 | Valid | 8,292.0 | — | 988 ms | — | — | 1 | — | — |
| Prefill | 64K | 1 | Valid | 8,205.0 | — | 7,813 ms | — | — | 1 | — | — |
| Sustained decode | 0 | 1 | Valid | 226.5 | 222.7 | 60 ms | 4.5 ms | 1.0 | 0 | 6,793 | 30.0 s |
| Sustained decode | 0 | 8 | Valid | 745.2 | 93.0 | 198 ms | 10.7 ms | 8.0 | 0 | 22,340 | 30.0 s |