Setup
This is a Zeus-wide tuning report anchored on the Unsloth Qwen3.8-27B GGUF page. The tables also include nine other model routes; each row identifies its own model and serving profile.
- Hardware: 2 × Intel Arc Pro B70 32 GB, AMD Ryzen 7 9700X, 32 GB system RAM, Windows 11.
- Runtime: llama.cpp b11190 at commit
fcc891545b0f06de346d8f67d1e6c61f9bf0e777, built in Release with oneAPI 2026.0,GGML_SYCL_F16=ON, andGGML_SYCL_DEVICE_ARCH=bmg_g21. - Driver: Intel Arc Pro 32.0.101.8805; Level Zero runtime 1.15.37858. The driver was not changed during this study.
- Comparison builds: b10488, b10621, or b10762 as listed per route. The new build was packaged with matching oneAPI runtime libraries and confirmed to use Level Zero.
- Dual-card routes used the default layer split with VMM off. One-card Gemma routes used VMM on. These are single-stream serving results, not aggregate concurrent throughput.
Prompt processing: matched production routes
Unless marked warmed, the comparison used one 2,013-token raw completion request and 128 generated tokens with the existing production model flags, temperature 0, and prompt caching disabled. Gemma tokenized the same text as 2,014 tokens. Warmed figures are repeat requests after the initial load. Speeds are tokens per second.
| Model route and profile | Prior build prompt | b11190 FP16 prompt | Generation observation |
|---|---|---|---|
| Qwen3.8 27B Q6 XL, one B70, 48k context, MTP3 | b10488: 260 | 482 | 33.1 → 34.7 |
| Qwen3.8 27B Q8_0, two B70s, 131k, MTP3 | b10488: 267 | 634 | 32.1 → 35.1 |
| Qwen3.8 27B Q6 XL, two B70s, 131k, MTP3 | b10488: 291 | 1,021 | 33.0 → 35.3 |
| GPT-OSS 120B UD-Q4_K_XL, two B70s, 131k | b10488: 501 | 986 | 30.8 → 30.5 |
| Huihui Qwen3.6 27B Q4_K, two B70s, 131k, MTP2, warmed | b10621: 343 | 1,168 | 29.1 → 38.1; same draft acceptance in matched test |
| Huihui Qwen3.6 35B A3B Q4_K, one B70, 131k, warmed | b10762: 1,747 | 2,360 | about 96 on both |
| Qwen3-Coder 30B A3B Q4_K_M, one B70, 40k, warmed | b10488: 1,794 | 2,474 | about 99 on both |
| HauhauCS Gemma 4 26B A4B Q4_K_M, one B70, 131k, MTP4, warmed | b10762: 1,398–1,402 | 2,075–2,119 | MTP acceptance varied |
| Official Gemma 4 26B A4B UD-Q4_K_XL, one B70, 262k, MTP4, warmed | b10762: 1,282–1,289 | 1,949–1,953 | MTP acceptance varied |
| Gemma 4 StyleTune V2 Q4_K_M, one B70, 262k, no MTP, warmed | b10762: 1,428–1,434 | 2,171–2,176 | 56.8 → 58.6 |
The Qwen3.8 Q6 one-card comparison above is the matched direct-server result. A later public-route verification measured 630 prompt tok/s, but it was not paired with another old-build run, so it is not used as the before/after figure.
Cold and warm behavior differed. The first FP16 request was slower for Qwen3-Coder and Huihui 35B despite their warmed gains. Hauhau cold results also varied substantially. Model-swap latency and first-request speed deserve separate measurement when a model is used only once per load.
Generation: n-gram drafting on predictable output
Each case requested 256 tokens after one warmup: novel scheduling reasoning, code refactoring, a minimal edit that copies most of an 80-line function, and technical-document summarization. Speeds below are generation tok/s. These are workload-specific results; n-gram drafting did not make arbitrary new text twice as fast.
| Model | Baseline novel / refactor / copied code / document | Kept n-gram setting | New novel / refactor / copied code / document |
|---|---|---|---|
| Qwen3-Coder 30B | 111 / 104 / 104 / 95 | --spec-type ngram-mod | 110 / 104 / 226 / 114 |
| Huihui Qwen3.6 35B | 101 / 98 / 99 / invalid | --spec-type ngram-mod | 100 / 98 / 214 / invalid |
| GPT-OSS 120B | 31.0 / 30.7 / 30.5 / 30.3 | --spec-type ngram-mod --spec-ngram-mod-n-match 48 | 31.1 / 30.9 / 36.9 / 30.6 |
The Huihui 35B document case ended after four tokens on both arms, so it is excluded. Coder had a repeat fresh-load run with the same result. The copied-code output was byte-for-byte identical between baseline and n-gram arms for all three models. A normal Coder coding answer was identical before and after the setting; GPT-OSS and Huihui 35B also passed chat smoke tests. These checks are not a comprehensive quality evaluation.
GPT-OSS needed the longer match: the default 24-token n-gram match improved copied code to 41.0 tok/s but slowed the refactor case from 30.7 to 28.3. A 32-token match still slowed it. The kept 48-token match preserved ordinary output speed while improving copied code by about 21%.
What did not win
- Huihui 27B MTP2 remained the balanced choice. MTP4 helped one document case but slowed novel output; MTP8 with
--spec-draft-p-min 0.8slowed all three tested cases. - One-card Qwen3.8 Q6 MTP6 raised copied-code output from 44.8 to 55.2 tok/s but lowered novel output from 31.2 to 26.9. Production remains MTP3.
- A local Qwen3.8 Q4_K_M file reached 20.74 raw single-token tok/s versus 18.38 for Q6 XL, yet its real MTP server was slower than Q6 across all four generation prompts. Q6 was retained.
- Disabling oneDNN flash attention produced 18.370 raw tok/s versus 18.378 at default. The older Level Zero adapter produced 18.244. Neither switch was retained.
- DeepSeek-R1-Distill-Llama-70B Q5_K_M at its 65k production context was worse on FP16: 11.8 → 4.7 prompt tok/s on a 458-token test. Decode was about 9 tok/s. It remains on b10488; its long-context prompt path warrants a separate investigation.
Reproduction notes and limits
Qwen3.8 Q6 one-card used 49,152 context, q4_0 K/V, batch 2,048, micro-batch 512, flash attention on, MTP3. The dual Qwen3.8 routes used 131,072 context and MTP3. Huihui 27B used q8_0 K/V and MTP2; Huihui 35B used q8_0 K/V without MTP. The three Gemma routes used f16 K/V; Hauhau and official IT used MTP4. GPT-OSS used -ngl auto, 131,072 context, and micro-batch 4,096.
Most before/after server points are one request per build; warmed Gemma, Huihui 35B, and Coder figures include repeats. Generation comparisons with MTP depend on draft acceptance. Raw repeated-text completions are timing fixtures, not an answer-quality benchmark. A two-chunk Qwen perplexity spot check differed by less than 0.1% between old and FP16 builds; that does not establish equivalence on every task. Long filled contexts, vision inputs, and sustained multi-user load were not measured here.
Current normal workload is Qwen3.8 Q6 on card 0 plus Hauhau Gemma on card 1. Both served requests on the new build with about 19 GB system RAM available.
Upstream references: llama.cpp b11190, SYCL backend guide, and speculative decoding guide.