Qwen3.8-27B on dual RTX 3060 12GB (llama.cpp)

Reproducible local decode numbers for two NVIDIA GeForce RTX 3060 12GB cards (24GB combined, no NVLink). Model is Unsloth Qwen3.8-27B-UD-Q4_K_XL. Engine is llama.cpp b10682.

Repo: github.com/Captain-Tripps/qwen38-dual-rtx-3060

Setup

GPUs: 2× RTX 3060 12GB, PCIe, no NVLink
CPU: Xeon E5-2690 v4, 64GB RAM, Ubuntu 24.04
Quant: UD-Q4_K_XL
Context: 65536, KV q4_0, text-only (no mmproj)
Speculation: native MTP only (--spec-type draft-mtp --spec-draft-n-max 2)
Split: --split-mode tensor --tensor-split 1,1
Env: GGML_CUDA_ALLREDUCE=internal

GGML_CUDA_ALLREDUCE=internal llama-server \
  -m Qwen3.8-27B-UD-Q4_K_XL.gguf \
  -ngl 99 \
  --split-mode tensor --tensor-split 1,1 \
  -c 65536 -ctk q4_0 -ctv q4_0 \
  -np 1 -fa on \
  --spec-type draft-mtp --spec-draft-n-max 2

Results (29 Aug 2026)

Chat /v1/chat/completions, thinking off, 256 max tokens. Decode tok/s from llama-server timings.

Config	Code tok/s	Prose tok/s
b10413 tensor + MTP	51.1	43.4
b10682 tensor + MTP (served)	57.0	45.5
b10682 layer + DFLASH2 n=7	36.3	16.1

VRAM on the winner: 9779 MiB per card.

What did not win

Official DFLASH2 (merged llama.cpp b10658) still cannot tensor-split. On layer split it is slower than MTP on this pair, especially prose (16 tok/s). MTP and DFLASH2 are alternative --spec-type values, not layers you stack.

Caveat for the leaderboard

These are served chat decode numbers with MTP on, not a no-spec llama-bench row. Short prompts, so prefill tok/s is not the story.

23 Aug at 147k on b10413: tensor+MTP was 44.8 / 35.3. Dropping to 64k and turning vision off is most of the 50+ tok/s jump; b10682 added the rest.

Attached evidenceLocalmaxxing runs referenced by this report — open any run to see the full submission

57.0 tok/sllama.cpp · UD-Q4_K_XL2× RTX 3060 · 12 GB each8/29/2026

Qwen3.8-27B on dual RTX 3060 12GB (llama.cpp tensor + MTP)