Speed test matrix
Qwen3.8-Flash-Next-W4A8-Paiton-RDNA4 on Radeon AI Pro R9700 32GB ×2
vllm · 0.29.0 · W4A81 context (0) × 2 concurrency levels (1–8) · prefill at 8K, 64K
Hardware
- Type
- Discrete GPU
- GPU
- Radeon AI Pro R9700
- GPU count
- 2
- VRAM
- 64 GB (2 × 32 GB)
- CPU
- AMD Ryzen Threadripper 3960X 24-Core Processor
- RAM
- 251 GB
- OS
- Linux
Engine
- Engine
- vllm
- Version
- 0.29.0
- Quantization
- W4A8
Run
- Served model
- Qwen3.8-Flash-Next
- Revision
- main
- Benchmark tool
- llm-inference-bench v0.7.7
- Decode test
- Fixed duration
- Prefill mode
- Measured during decode warm-up
- Time per measurement
- 30 s
- Max output
- 8,192 tokens
Prefill
prompt tok/s by context length8K context7,1211,151 ms TTFT
64K context6,9949,166 ms TTFT
Sustained decode
total tok/s · rows are context length, columns are simultaneous requests| Context | 1 | 8 |
|---|---|---|
| 0 | 199.867 ms TTFT | 678.2214 ms TTFT |
TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.
All measurements
Latencies are medians. Total tok/s is output across all simultaneous requests.| Phase | Context | Requests at once | Status | Total tok/s | Per request tok/s | TTFT | Time between tokens | Average active requests | Completed requests | Output tokens | Measured for |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Prefill | 8K | 1 | Valid | 7,121.0 | — | 1,151 ms | — | — | 1 | — | — |
| Prefill | 64K | 1 | Valid | 6,994.0 | — | 9,166 ms | — | — | 1 | — | — |
| Sustained decode | 0 | 1 | Valid | 199.8 | 200.1 | 67 ms | 5.0 ms | 1.0 | 0 | 5,993 | 30.0 s |
| Sustained decode | 0 | 8 | Valid | 678.2 | 86.3 | 214 ms | 11.6 ms | 8.0 | 0 | 20,345 | 30.0 s |