Qwen3.8-Flash-Next Unsloth UD-IQ4_XS on dual 16 GB (RTX 5080 + RTX 5060 Ti) — Strata v0.1.43.1 speed report
Summary: Canonical speed suite on the live endpoint, measured 2026-10-13 on Strata v0.1.43.1 with the user's recalibrated UD-IQ4_XS pack: reasoning-v1 105.7 tok/s, code-v1 107.6 tok/s (TTFT 649.5 / 589.4 ms), verified-at-depth decode 101.4 / 100.8 / 94.3 tok/s at 16k / 65k / 131k with cold prefill 3,233 / 3,679 / 3,618 tok/s (client; engine 3,250 / 3,682 / 3,617). Submitted 2026-10-12T19:56Z (= 2026-10-13 02:56 local); server read-back confirms 5/5 runs verifiedRun: true.
Setup
- Model:
unsloth/Qwen3.8-Flash-Next-GGUFUD-IQ4_XS (180B MoE, 3-shard GGUF), served asqwen3.8-flash-next-unsloth-ud-iq4_xs, 262,144 context, vision on, no control vector on this pack. - Engine: Strata v0.1.43.1, commit
a0116b2782a5b82622358c04a06496485d12b2d6— hotfix for v0.1.43; answers byte-identical to v0.1.43 per release identity tests. Launch flags verbatim from the live process (paths abbreviated; full snippet in each run file'sengineFlags.commandSnippet):
~/Projects/Strata/engine/strata --serve --pack ~/Projects/Strata-data/packs/unsloth-ud-iq4_xs --native ~/Projects/Strata-data/models/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf --expert-profile ~/Projects/Strata/data/expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp ~/Projects/Strata-data/mtp/rt --max-context 262144 --kv int8 --kv-resident 32768 --resident-budget-gib 55 --vision --vram-reserve-mib 700 --spec-min-p 0.5 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97 --layer-split auto
- No
STRATA_*env; PCIe share auto-measured 0.55; adaptive overlap OFF per start-up log (adaptive overlap: off: not on a layer split— for this pack--layer-split autoresolves to no split; the second card extends the expert cache). - Hardware: RTX 5080 16 GB + RTX 5060 Ti 16 GB, Ryzen 7 7800X3D, 96 GB DDR5, Arch Linux.
- Tooling: localmaxxing CLI v0.1.51; engine name
custom+ full provenance. Protocol: 2 warmups + 3 timed streaming iterations, median; temp 0, batch 1, cache-bust nonce per request (cold prefill), 512 output tokens, inter-token decode window, reasoning disabled (reasoning_effort: none), 2 s nvidia-smi peak sampler. lss-collector parked for the whole measurement window; engine args re-verified at chain start and end (identical).
Speed — canonical prompts (verified-eligible)
| Metric | reasoning-v1 | code-v1 |
|---|---|---|
| Decode (median) | 105.7 tok/s (105.7/102.0/107.6) | 107.6 tok/s (102.0/107.6/107.6) |
| TTFT (median) | 649.5 ms (647.6/649.5/658.0) | 589.4 ms (627.2/589.4/588.5) |
| Prefill, client-estimated | 435.7 tok/s | 369.9 tok/s |
| MTP acceptance | 84.4% | 80.7% |
| Peak VRAM / power | 30.2 GB / 153+54 W | 30.1 GB / 153+53 W |
Measurement history (full disclosure): two earlier reasoning-v1 passes were contaminated by a /review subagent dispatched 37 s after a re-run launch — it ran inference on the same single-slot engine, its growing context visible in the engine request history as a 54k→156k-token series (19:16–19:22Z). Pass 1 kept only via clean iterations 2–3; the 19:14Z re-run was fully contaminated (TTFT 30.7–45.6 s) and discarded; both preserved as data/*-contaminated-20261012T191{2,9}Z.json. The submitted pass (19:54Z) is fully clean: TTFT spread 1.003× median-vs-best, passing the TTFT sanity gate now built into the measurers. An interim reasoning row submitted at 19:56Z carried the same clean numbers (105.7/649.5 — verified by reading the server row) but a stale hardcoded disclosure note describing the old discarded passes; the note was made conditional and the row resubmitted at 19:58Z. The interim row cmv5o9z830049p001wmaofp40 first remained as a verified duplicate — its stale note corrected via owner edit (PATCH /api/runs/{id}, ~20:35Z) — then, after independent review flagged the double-count risk on the leaderboard, was deleted via owner DELETE /api/runs/{id} (2026-10-13, HTTP 204). The five rows above are now the complete verified UD-IQ4_XS set.
Verified-at-depth (canonical reasoning-v1 as final turn, cold prefill)
| Level | Decode (median) | TTFT | Prefill client / engine | Acceptance |
|---|---|---|---|---|
| 16k | 101.4 (105.1/101.4/94.8) | 4,974 ms | 3,233 / 3,250 tok/s | 88.3% |
| 65k | 100.8 (100.8/99.8/102.5) | 17,692 ms | 3,679 / 3,682 tok/s | 79.2% |
| 131k | 94.3 (97.6/94.3/90.0) | 35,893 ms | 3,618 / 3,617 tok/s | 84.6% |
Peak VRAM 30.11 GB and power 259+84 W were sampled across the full 3-level session, not per level. All nine at-depth TTFTs are within ±1% — no contamination at depth.
vs the approved UD-Q4_K_XL runs (v0.1.43 config B, same rig)
| Metric | UD-IQ4_XS (this report) | UD-Q4_K_XL config B (approved) | Δ |
|---|---|---|---|
| reasoning-v1 decode | 105.7 | 98.1 | +7.7% |
| code-v1 decode | 107.6 | 102.8 | +4.7% |
| TTFT reasoning / code | 649.5 / 589.4 ms | 866 / 770 ms | −25% / −23% |
| at-depth 16k / 65k / 131k | 101.4 / 100.8 / 94.3 | 82.0 / 88.0 / 89.7 | +24% / +15% / +5% |
Mixed causes — do not attribute to any single change. The two configs differ in quantization (IQ4_XS streams fewer bytes per expert hit), control vector (absent here), resident budget (55 vs 72 GiB), split mode (auto/no-split vs --layer-split 43 + STRATA_ROUTE_RESIDENT=1), and engine hotfix (.43.1 vs .43, answers identical). The direction is consistent across all five suites: the IQ4_XS pack is faster on this rig at every context depth, with the biggest gain where PCIe streaming pressure is highest (16k).
Artifacts
- Run files (all dry-run
verified: true, zero issues, privacy-scrubbed — 0 username occurrences):runs/unsloth-Qwen3.8-Flash-Next-GGUF/v01431-{reasoning-v1,code-v1,verified-at-depth-{16384,65536,131072}}.json - Raw measurements:
data/reasoning-off-{reasoning-v1,code-v1}.json,data/verified-at-depth.json; contaminated passes preserved as*-pass1-contaminated-20261012T1912Z.json,*-rerun-fully-contaminated-20261012T1919Z.json(UTC tags) - Engine provenance snapshot:
data/probes-20261013/provenance-v01431-iq4xs.txt - Chain log (full protocol trail incl. dry-run gates):
data/v01431-speed-chain.log - Scripts:
measure-reasoning-off.py,measure-verified-at-depth.py(served model now derived from/healthat run time),build-v01431-canonical.py,build-v01431-at-depth.py,run-v01431-speed-chain.sh
Reproduce
cd ~/Projects/Benchmark/localmaxxing
./run-v01431-speed-chain.sh # drain 75 s → arg guard → 2 suites → at-depth → build → dry-run ×5 (no submit)
./lmx speed-test dry-run runs/unsloth-Qwen3.8-Flash-Next-GGUF/v01431-reasoning-v1.json
Submission status: SUBMITTED & VERIFIED. All five payloads submitted 2026-10-12T19:56Z (= 2026-10-13 02:56 local) after the verified-gate ×5 passed; server read-back (/api/speed-tests, authenticated) confirms 5/5 verifiedRun: true: code-v1 cmv5o9wvr0043p001ling4vjz (107.6), reasoning-v1 corrected cmv5oced4004xp001bdmgqvs0 (105.7 / 649.5 ms), 16k cmv5oa3kw004lp0012hmhg13y (101.4), 65k cmv5oa5x4004rp001gxy4aqqo (100.8), 131k cmv5oa1na004fp001bnovi3la (94.3). One duplicate interim reasoning row (cmv5o9z830049p001wmaofp40) was deleted on 2026-10-13 via owner DELETE /api/runs/{id} (HTTP 204; read-back confirms verified IQ4_XS rows are exactly the five listed above).