Reports

Community-authored local inference reports

Published field reportsnewest

4x Arc Pro B65: context sweep 0-200K, 12 models, vLLM + llama.cpp

Decode and prefill from context 0 to 200K at 1-16 users on 4x Intel Arc Pro B65 (and 1 card): 30 vLLM matrices across 12 models, Qwen3.8-Flash-Next prefill chunk 2048 vs 8192, and three llama.cpp depth sweeps. Every number links to its LocalMaxxing matrix or run.

Strata dual-16 GB (5080+5060 Ti): verified 98.1/102.8 tok/s, 262k context, 3 full evals — Unsloth UD-Q4_K_XL

Qwen3.8-Flash-Next-GGUF

Complete field report on Strata v0.1.42–v0.1.43: 15 verified speed/depth rows (canonical prompts, cold prefill, MTP), and three full quality evals — HellaSwag 92.66%, GSM8K 98.10%, ARC-Challenge 98.43% — all 13/13 shards approved. 180B MoE on 2x16 GB consumer GPUs via expert streaming.

Q4_K_XL custom2× Multi-GPU · 32 GB

15 speed tests

rtx-5080 multi-gpu speculative-decoding expert-streaming moe long-context evals

Strata dual-16 GB (5080+5060 Ti): verified 105.7/107.6 tok/s, 262k context, cold prefill 3.2–3.7k — UD-IQ4_XS

Qwen3.8-Flash-Next-GGUF

Verified UD-IQ4_XS speed suite on Strata v0.1.43.1: reasoning 105.7 / code 107.6 tok/s (TTFT 649/589 ms), verified-at-depth decode 101.4/100.8/94.3 tok/s at 16k/65k/131k with cold prefill 3.2–3.7k tok/s. All five runs verifiedRun:true; full contamination-handling disclosure included.

IQ4_XS custom2× Multi-GPU · 32 GB

5 speed tests

rtx-5080 multi-gpu speculative-decoding expert-streaming moe long-context

Qwen3.8-27B long-context test on dual RTX 3060 12GB

Qwen3.8-27B

Qwen3.8-27B on two RTX 3060 12GB cards, llama.cpp 0.4.1-dev. Measured 129,000-token synthetic prompt at 497.8 prompt tok/s and 42.7 output tok/s mean, with exact recall of 3/3 canary facts in two runs. Configured context 131,072; ngram-mod + MTP3; batch 2048, ubatch 512.

long-context throughput multi-gpu rtx-3060 llama.cpp qwen3.8

Zeus 2× Arc Pro B70: llama.cpp SYCL FP16 tuning across Qwen3.8 and nine other model routes

Qwen3.8-27B-GGUF

On two Intel Arc Pro B70s, a b11190 SYCL FP16 build substantially improved prompt processing across 10 production routes. This report gives matched serving results, copy-heavy generation gains from n-gram drafting, rejected tuning variants, and measurement limits.

Q6_K_XL focus; Q8_0, Q4_K_M, UD-Q4_K_XL llama.cpp b11190, SYCL / Level Zero

intel-arc battlemage sycl llama.cpp fp16 multi-gpu speculative-decoding throughput

DeepSeek-V4.1-Flash on 4x DGX Spark: 76 tok/s verified

DeepSeek-V4.1-Flash

Verified single-stream run of deepseek-ai/DeepSeek-V4.1-Flash on 4x NVIDIA GB10 (DGX Spark, 512 GB unified, TP4, SGLang, DSPARK spec decoding): 76.0 tok/s decode, 887.6 tok/s prefill, 308.7 ms TTFT, 423 GB unified memory in use, 37.2% draft acceptance.

FP8+MXFP4 sglangGB10 Grace Blackwell · 512 GB

1 speed test

throughput multi-gpu unified-memory sglang moe speculative-decoding dgx-spark fp8

Qwen3.6-35B-A3B on dual RTX 3060 12GB (llama.cpp tensor + MTP, 88k)

Qwen3.6-35B-A3B-GGUF

Two RTX 3060 12GB, no NVLink. llama.cpp b10682, Unsloth UD-Q4_K_XL, 88k. Tensor split + native MTP + q4_0 draft KV: 129 tok/s code / 104 tok/s prose. 96k+MTP does not fit.

1 speed test

throughput multi-gpu rtx-3060 llama.cpp qwen3.6 moe

Qwen3.8-27B on dual RTX 3060 12GB (llama.cpp tensor + MTP)

Qwen3.8-27B-GGUF

Two RTX 3060 12GB, no NVLink. llama.cpp b10682, Unsloth UD-Q4_K_XL, 64k text-only. Tensor split + native MTP: 57 tok/s code / 46 tok/s prose. DFLASH2 on layer split lost.

1 speed test

throughput multi-gpu rtx-3060 llama.cpp qwen3.8