--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B at the short-prompt cell: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 516 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 383.2 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5435 from measured-window counters (804 draft / 437 accepted). POWER: 230 W CONFIGURED CAP; measured avg draw 224.53 W (xe hwmon energy1_input 3277950683 uJ over 14.599 s, bracketing the measured window only); 5.12 J/token over that window INCLUDING prefill; tokens/J 0.1952 (vs 0.2566 patched = +31.4% for the patched arm). RESULT 52.61 vs 73.43 tok/s for the patched arm in this same cell = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 516 = 0.19%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT at the short-prompt cell: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 517 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 352.0 ms (baseline 383.2 ms - TTFT also improves in this cell). tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5581 from measured-window counters (792 draft / 442 accepted); run-variable across runs, this cell's patched medians 73.43 (n=5) and 77.14 (n=3 screen), samples 73.4-88.7 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 228.60 W (xe hwmon energy1_input 2493763306 uJ over 10.909 s, bracketing the measured window only); 3.90 J/token over that window INCLUDING prefill; tokens/J 0.2566 vs 0.1952 baseline = +31.4%. RESULT 73.43 vs 52.61 tok/s = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 517 = 0.19%); single variable = the two draft-INT4 env gates. Same-host re-runs +48.6% (p8192), +37.2/+44.4% (n=5), +46.0% (n=3 screen). Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8190 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4738.1 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.4873 from measured-window counters (864 draft / 421 accepted); run-variable, 0.4873 this run vs 0.53-0.58 in the earlier baseline run - this run's baseline arm is at the low tail of acceptance and throughput, so the 1.49x here is the HIGH end of the reproduced +37.2/+44.4/+48.6% range. POWER: 230 W CONFIGURED CAP; measured avg draw 228.74 W (xe hwmon energy1_input 10730884522 uJ over 46.914 s, bracketing the measured window only); 16.77 J/token over that window INCLUDING prefill; tokens/J 0.0596 (vs 0.0654 patched = +9.6% for the patched arm). RESULT 48.81 vs 72.55 tok/s for the patched arm in this same cell = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8190 = 0.05%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8194 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4791.6 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5676 from measured-window counters (784 draft / 445 accepted); run-variable across same-host runs 0.55-0.70 (this 0.5676); patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 229.50 W (xe hwmon energy1_input 9788913941 uJ over 42.654 s, bracketing the measured window only); 15.30 J/token over that window INCLUDING prefill; tokens/J 0.0654 vs 0.0596 baseline = +9.6%. RESULT 72.55 vs 48.81 tok/s = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8194 = 0.05%); single variable = the two draft-INT4 env gates. Same-host re-runs +39.6% (p512), +37.2/+44.4% (n=5), +46.0% (n=3 screen); a both-arms-patched control gave 80.61/78.27 tok/s. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B at the short-prompt cell: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 516 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 383.2 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5435 from measured-window counters (804 draft / 437 accepted). POWER: 230 W CONFIGURED CAP; measured avg draw 224.53 W (xe hwmon energy1_input 3277950683 uJ over 14.599 s, bracketing the measured window only); 5.12 J/token over that window INCLUDING prefill; tokens/J 0.1952 (vs 0.2566 patched = +31.4% for the patched arm). RESULT 52.61 vs 73.43 tok/s for the patched arm in this same cell = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 516 = 0.19%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT at the short-prompt cell: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 517 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 352.0 ms, improved vs baseline 383.2 ms (NOT unchanged). tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5581 from measured-window counters (792 draft / 442 accepted); run-variable 0.55-0.70 across same-host runs; patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 228.59 W (xe hwmon energy1_input 2493763306 uJ over 10.909 s, bracketing the measured window only); 3.89 J/token over that window INCLUDING prefill; tokens/J 0.2566 vs 0.1952 baseline = +31.4%. RESULT 73.43 vs 52.61 tok/s = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 517 = 0.19%); single variable = the two draft-INT4 env gates. Same-host re-runs +48.6% (p8192), +37.2/+44.4% (n=5), +46.0% (n=3 screen). Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8190 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4738.1 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.4873 from measured-window counters (864 draft / 421 accepted); run-variable, 0.4873 this run vs 0.53-0.58 in the earlier baseline run - this run's baseline arm is at the low tail of acceptance and throughput, so the 1.49x here is the HIGH end of the reproduced +37.2/+44.4/+48.6% range. POWER: 230 W CONFIGURED CAP; measured avg draw 228.74 W (xe hwmon energy1_input 10730884522 uJ over 46.914 s, bracketing the measured window only); 16.77 J/token over that window INCLUDING prefill; tokens/J 0.0596 (vs 0.0654 patched = +9.6% for the patched arm). RESULT 48.81 vs 72.55 tok/s for the patched arm in this same cell = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8190 = 0.05%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8194 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4791.6 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5676 from measured-window counters (784 draft / 445 accepted); run-variable across same-host runs 0.55-0.70 (this 0.5676); patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 229.49 W (xe hwmon energy1_input 9788913941 uJ over 42.654 s, bracketing the measured window only); 15.29 J/token over that window INCLUDING prefill; tokens/J 0.0654 vs 0.0596 baseline = +9.6%. RESULT 72.55 vs 48.81 tok/s = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8194 = 0.05%); single variable = the two draft-INT4 env gates. Same-host re-runs +39.6% (p512), +37.2/+44.4% (n=5), +46.0% (n=3 screen); a both-arms-patched control gave 80.61/78.27 tok/s. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
Prefill record (separate from decode record by design: different binary/spec). Quant: AtomicChat AD-4.27bpw Q4_K_M M64 GGUF. F16-accumulation binary is worth ~+38% vs FP32 at b3072. Self-measured engine-native C1, single client, n=3 median, P9096(actual)/G128/C1 COLD INPUT rate (llama.cpp HTTP full uncached prompt, not llama-bench pp), zero prompt-cache reuse verified per sample, stdev 0.2. Decode in same cell 22.96 tok/s (no-spec F16). PLE table mmap host. PROVISIONAL self-report, not independently reproduced.
C1 only (-np 1, one client). c16 means context 16384, not concurrency 16. Prefill = llama.cpp engine timings.prompt_per_second median n=5 at actual p9096/g128, fused IQ3_S/IQ4_NL MMVQ, GGML_SYCL_F16=ON, -b/-ub 3072, cache_n=0. Decode on this same F16 server is 20.34, not the fused-FP32 23.38 cell. Artifact AtomicChat Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64 rev 142262902a46f7daed19c79d0771534c8106ad59. Mixed: gate/up IQ3_S, down IQ4_NL, PLE Q5_1 CPU mmap (platform quantization field is IQ3_S). KV q8_0/q4_1, FA on, layer split 1,1, 195 W configured cap. Loaded-idle GPU1 pkg ~62 C. E2 self-report. Independent reproduction pending. Patches: patches/llamacpp-sycl/flashnext-arch-overlay.patch then sycl-fused-mmvq-and-gpu-group.patch.
C1 only (-np 1, one client). c8 means context 8192, not concurrency 8. Decode = llama.cpp engine timings.predicted_per_second median n=5 at p512/g128, fused IQ3_S/IQ4_NL MMVQ, GGML_SYCL_F16=OFF. Cold input = timings.prompt_per_second with cache_n=0 (183.98). Do not mix this decode number with the fused-F16 594.49 prefill cell (separate binary). Artifact AtomicChat Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64 rev 142262902a46f7daed19c79d0771534c8106ad59. Mixed: gate/up IQ3_S, down IQ4_NL, PLE Q5_1 CPU mmap (platform quantization field is IQ3_S). KV q8_0/q4_1, FA on, layer split 1,1, 195 W configured cap. Loaded-idle GPU1 pkg ~62 C, not ordinary <=55 C. E2 self-report. Independent reproduction pending. Patches: patches/llamacpp-sycl/flashnext-arch-overlay.patch then sycl-fused-mmvq-and-gpu-group.patch.
Self-reported. Single Intel Arc Pro B70 32GB. Realistic 5-user coding serving, v2 with REAL completions kept in session history (supersedes the stubbed same-day 138.3 record). Stack: vLLM XPU nightly f01e24f6, MTP4, GDN mixed-split v5, draft-INT4 S+M1, prefix caching ON. 5 concurrent sessions, ~8K start, 3 turns, g512, 60/60 OK. tokSOut = sum of per-stream generation rates (5 x 25.5 median per-user tok/s, all turns). Per-turn: t1 165.5, t2 127.9, t3 114.2 aggregate. TTFT 22.6-25.0s every turn: prefix cache lands 0-38% of shared tokens at C5 on this build (vs 91% at C1), so ~45-53K session tokens re-prefill per turn. MTP acceptance 43-56% under concurrency. Short-prompt C5 (203.8) is a separate record; C1 is 106.7.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B at the short-prompt cell: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 516 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 383.2 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5435 from measured-window counters (804 draft / 437 accepted). POWER: 230 W CONFIGURED CAP; measured avg draw 224.53 W (xe hwmon energy1_input 3277950683 uJ over 14.599 s, bracketing the measured window only); 5.12 J/token over that window INCLUDING prefill; tokens/J 0.1952 (vs 0.2566 patched = +31.4% for the patched arm). RESULT 52.61 vs 73.43 tok/s for the patched arm in this same cell = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 516 = 0.19%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT at the short-prompt cell: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 517 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 352.0 ms (baseline 383.2 ms - TTFT also improves in this cell). tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5581 from measured-window counters (792 draft / 442 accepted); run-variable across runs, this cell's patched medians 73.43 (n=5) and 77.14 (n=3 screen), samples 73.4-88.7 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 228.60 W (xe hwmon energy1_input 2493763306 uJ over 10.909 s, bracketing the measured window only); 3.90 J/token over that window INCLUDING prefill; tokens/J 0.2566 vs 0.1952 baseline = +31.4%. RESULT 73.43 vs 52.61 tok/s = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 517 = 0.19%); single variable = the two draft-INT4 env gates. Same-host re-runs +48.6% (p8192), +37.2/+44.4% (n=5), +46.0% (n=3 screen). Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8190 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4738.1 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.4873 from measured-window counters (864 draft / 421 accepted); run-variable, 0.4873 this run vs 0.53-0.58 in the earlier baseline run - this run's baseline arm is at the low tail of acceptance and throughput, so the 1.49x here is the HIGH end of the reproduced +37.2/+44.4/+48.6% range. POWER: 230 W CONFIGURED CAP; measured avg draw 228.74 W (xe hwmon energy1_input 10730884522 uJ over 46.914 s, bracketing the measured window only); 16.77 J/token over that window INCLUDING prefill; tokens/J 0.0596 (vs 0.0654 patched = +9.6% for the patched arm). RESULT 48.81 vs 72.55 tok/s for the patched arm in this same cell = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8190 = 0.05%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8194 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4791.6 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5676 from measured-window counters (784 draft / 445 accepted); run-variable across same-host runs 0.55-0.70 (this 0.5676); patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 229.50 W (xe hwmon energy1_input 9788913941 uJ over 42.654 s, bracketing the measured window only); 15.30 J/token over that window INCLUDING prefill; tokens/J 0.0654 vs 0.0596 baseline = +9.6%. RESULT 72.55 vs 48.81 tok/s = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8194 = 0.05%); single variable = the two draft-INT4 env gates. Same-host re-runs +39.6% (p512), +37.2/+44.4% (n=5), +46.0% (n=3 screen); a both-arms-patched control gave 80.61/78.27 tok/s. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B at the short-prompt cell: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 516 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 383.2 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5435 from measured-window counters (804 draft / 437 accepted). POWER: 230 W CONFIGURED CAP; measured avg draw 224.53 W (xe hwmon energy1_input 3277950683 uJ over 14.599 s, bracketing the measured window only); 5.12 J/token over that window INCLUDING prefill; tokens/J 0.1952 (vs 0.2566 patched = +31.4% for the patched arm). RESULT 52.61 vs 73.43 tok/s for the patched arm in this same cell = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 516 = 0.19%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window; 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT at the short-prompt cell: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 517 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 352.0 ms, improved vs baseline 383.2 ms (NOT unchanged). tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5581 from measured-window counters (792 draft / 442 accepted); run-variable 0.55-0.70 across same-host runs; patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 228.59 W (xe hwmon energy1_input 2493763306 uJ over 10.909 s, bracketing the measured window only); 3.89 J/token over that window INCLUDING prefill; tokens/J 0.2566 vs 0.1952 baseline = +31.4%. RESULT 73.43 vs 52.61 tok/s = 1.40x (+39.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 1 tok of 517 = 0.19%); single variable = the two draft-INT4 env gates. Same-host re-runs +48.6% (p8192), +37.2/+44.4% (n=5), +46.0% (n=3 screen). Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p512-n5-...-20260922T214938Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
BASELINE ARM of a matched_with_tokenizer_delta A/B: GPTQ-INT4 g128 sym target + BF16 MTP4 drafter, drafter NOT quantized. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8190 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4738.1 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.4873 from measured-window counters (864 draft / 421 accepted); run-variable, 0.4873 this run vs 0.53-0.58 in the earlier baseline run - this run's baseline arm is at the low tail of acceptance and throughput, so the 1.49x here is the HIGH end of the reproduced +37.2/+44.4/+48.6% range. POWER: 230 W CONFIGURED CAP; measured avg draw 228.74 W (xe hwmon energy1_input 10730884522 uJ over 46.914 s, bracketing the measured window only); 16.77 J/token over that window INCLUDING prefill; tokens/J 0.0596 (vs 0.0654 patched = +9.6% for the patched arm). RESULT 48.81 vs 72.55 tok/s for the patched arm in this same cell = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8190 = 0.05%); single variable = the two draft-INT4 env gates. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class; every emitted token target-verified. ENABLER for the patched effect: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM issue #54785 (2026-09-01) reports capture + MTP num_speculative_tokens=4 can give non-deterministic wrong logits at temperature=0 (k<=3 clean); both arms ran capture-on. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
--presence-penalty 1.5; one same-shape warmup discarded; prefix-cache hit delta zero across the measured window (unique cold prompt with entropy nonce); 10/10 requests finished_reason=length at exactly 128 tokens
Notes
THE IMPROVEMENT: draft-side INT4. Only the MTP drafter's weights are quantized (its 2.54 GB F16 LM head + the MTP body's 5 linears -> W4A16 g128 sym); the 27B target and its verification logits path are untouched. Metric: client post-first streaming rate (request-side), median of n=5 post-warmup; NOT engine-native tg. Coordinates: C1, actual 8194 prompt tokens (server tokenizer), g128 all finish_reason=length (640/640), cold_model_cold_prefix (prefix caching disabled, zero hit delta). TTFT median 4791.6 ms. tokSPrefill derived = promptTokens/TTFT (cold input rate = actual endpoint prompt tokens / client TTFT). Acceptance 0.5676 from measured-window counters (784 draft / 445 accepted); run-variable across same-host runs 0.55-0.70 (this 0.5676); patched medians span 72.55-80.32, within-run samples 64.25-91.17 (band, not a point). POWER: 230 W CONFIGURED CAP; measured avg draw 229.49 W (xe hwmon energy1_input 9788913941 uJ over 42.654 s, bracketing the measured window only); 15.29 J/token over that window INCLUDING prefill; tokens/J 0.0654 vs 0.0596 baseline = +9.6%. RESULT 72.55 vs 48.81 tok/s = 1.49x (+48.6%), n=5/arm median. Class matched_with_tokenizer_delta (arms' prompts differ 4 tok of 8194 = 0.05%); single variable = the two draft-INT4 env gates. Same-host re-runs +39.6% (p512), +37.2/+44.4% (n=5), +46.0% (n=3 screen); a both-arms-patched control gave 80.61/78.27 tok/s. Correctness: greedy byte-identical patched vs unpatched (3 prompts x 2 repeats); 30-draw x 6-prompt identical modal emitted class. ENABLER: XPU graph capture REQUIRED (capture off = inert <=1.5%; capture alone 1.68x); upstream vLLM #54785 reports capture+MTP4 can give non-deterministic wrong logits at temp=0 (k<=3 clean); both arms capture-on so the A/B is consistent. Stack uses local unpublished overlay patches; the image digest is public and pullable. PROVISIONAL self-report. Evidence: results/2026/09/b70-improvement-probes-20260920/p4-champion-ab-p8192-n5-...-20260922T213957Z/.
Prefill record (separate from decode record by design: different binary/spec). Quant: AtomicChat AD-4.27bpw Q4_K_M M64 GGUF. F16-accumulation binary is worth ~+38% vs FP32 at b3072. Self-measured engine-native C1, single client, n=3 median, P9096(actual)/G128/C1 COLD INPUT rate (llama.cpp HTTP full uncached prompt, not llama-bench pp), zero prompt-cache reuse verified per sample, stdev 0.2. Decode in same cell 22.96 tok/s (no-spec F16). PLE table mmap host. PROVISIONAL self-report, not independently reproduced.
C1 only (-np 1, one client). c16 means context 16384, not concurrency 16. Prefill = llama.cpp engine timings.prompt_per_second median n=5 at actual p9096/g128, fused IQ3_S/IQ4_NL MMVQ, GGML_SYCL_F16=ON, -b/-ub 3072, cache_n=0. Decode on this same F16 server is 20.34, not the fused-FP32 23.38 cell. Artifact AtomicChat Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64 rev 142262902a46f7daed19c79d0771534c8106ad59. Mixed: gate/up IQ3_S, down IQ4_NL, PLE Q5_1 CPU mmap (platform quantization field is IQ3_S). KV q8_0/q4_1, FA on, layer split 1,1, 195 W configured cap. Loaded-idle GPU1 pkg ~62 C. E2 self-report. Independent reproduction pending. Patches: patches/llamacpp-sycl/flashnext-arch-overlay.patch then sycl-fused-mmvq-and-gpu-group.patch.
C1 only (-np 1, one client). c8 means context 8192, not concurrency 8. Decode = llama.cpp engine timings.predicted_per_second median n=5 at p512/g128, fused IQ3_S/IQ4_NL MMVQ, GGML_SYCL_F16=OFF. Cold input = timings.prompt_per_second with cache_n=0 (183.98). Do not mix this decode number with the fused-F16 594.49 prefill cell (separate binary). Artifact AtomicChat Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64 rev 142262902a46f7daed19c79d0771534c8106ad59. Mixed: gate/up IQ3_S, down IQ4_NL, PLE Q5_1 CPU mmap (platform quantization field is IQ3_S). KV q8_0/q4_1, FA on, layer split 1,1, 195 W configured cap. Loaded-idle GPU1 pkg ~62 C, not ordinary <=55 C. E2 self-report. Independent reproduction pending. Patches: patches/llamacpp-sycl/flashnext-arch-overlay.patch then sycl-fused-mmvq-and-gpu-group.patch.
Self-reported. Single Intel Arc Pro B70 32GB. Realistic 5-user coding serving, v2 with REAL completions kept in session history (supersedes the stubbed same-day 138.3 record). Stack: vLLM XPU nightly f01e24f6, MTP4, GDN mixed-split v5, draft-INT4 S+M1, prefix caching ON. 5 concurrent sessions, ~8K start, 3 turns, g512, 60/60 OK. tokSOut = sum of per-stream generation rates (5 x 25.5 median per-user tok/s, all turns). Per-turn: t1 165.5, t2 127.9, t3 114.2 aggregate. TTFT 22.6-25.0s every turn: prefix cache lands 0-38% of shared tokens at C5 on this build (vs 91% at C1), so ~45-53K session tokens re-prefill per turn. MTP acceptance 43-56% under concurrency. Short-prompt C5 (203.8) is a separate record; C1 is 106.7.
