SergiioB

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

69.3 tok/s

Prefill

1,755 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

—

RAM

—

OS

Linux

Power

—

Engine

vllm

Engine version

0.26.1rc1.dev457+gc810e5ee9.xpu

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

0.88

Max running seqs

64

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 131072 --gpu-memory-utilization 0.88 --kv-cache-dtype fp8 --max-num-seqs 64 --max-num-batched-tokens 8192 --enable-prefix-caching --language-model-only --speculative-config {"method":"mtp","num_speculative_tokens":4}

Extra flags

--max-model-len 131072 --language-model-only --speculative-config {method:mtp,num_speculative_tokens:4}

Notes

Single-stream C1. vLLM XPU pinned nightly (vllm/vllm-openai-xpu@sha256:2c427ef477da092eb6f2cdbbbd24950b5fa171565b916db69d4c7bb10e68ca97), vllm-xpu-kernels 0.1.12. Model: llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GPTQ-Int4 (dense 27B, GPTQ-INT4 g128 sym, preserved BF16 MTP head). Patches: patch_mtp_nightly.py + patch_mtp_boundary.py. fp8 KV cache (REQUIRED for dense 128K; fp16 does not fit). MTP4 speculative decode, 1-layer draft. gpu-memory-utilization 0.88, scheduler 8192, max-num-seqs 64, prefix cache on with zero hit delta. tokSOut = client post-first rate at p512/g128, median n=5 (not engine-native decode); tokSPrefill = actual input tokens / client TTFT at p2048/g1, median n=5 (not llama-bench pp). 230W configured cap. E2 provisional self-reported; independent reproduction pending. Measured 2026-08-09 Run 31.

Reactions

—

Submitted

Aug 9, 2026, 10:57 PM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

69.3 tok/s

Prefill

1,755 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

—

RAM

—

OS

Linux

Power

—

Engine

vllm

Engine version

0.26.1rc1.dev457+gc810e5ee9.xpu

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

0.88

Max running seqs

64

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 131072 --gpu-memory-utilization 0.88 --kv-cache-dtype fp8 --max-num-seqs 64 --max-num-batched-tokens 8192 --enable-prefix-caching --language-model-only --speculative-config {"method":"mtp","num_speculative_tokens":4}

Extra flags

--max-model-len 131072 --language-model-only --speculative-config {method:mtp,num_speculative_tokens:4}

Notes

Single-stream C1. vLLM XPU pinned nightly (vllm/vllm-openai-xpu@sha256:2c427ef477da092eb6f2cdbbbd24950b5fa171565b916db69d4c7bb10e68ca97), vllm-xpu-kernels 0.1.12. Model: llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GPTQ-Int4 (dense 27B, GPTQ-INT4 g128 sym, preserved BF16 MTP head). Patches: patch_mtp_nightly.py + patch_mtp_boundary.py. fp8 KV cache (REQUIRED for dense 128K; fp16 does not fit). MTP4 speculative decode, 1-layer draft. gpu-memory-utilization 0.88, scheduler 8192, max-num-seqs 64, prefix cache on with zero hit delta. tokSOut = client post-first rate at p512/g128, median n=5 (not engine-native decode); tokSPrefill = actual input tokens / client TTFT at p2048/g1, median n=5 (not llama-bench pp). 230W configured cap. E2 provisional self-reported; independent reproduction pending. Measured 2026-08-09 Run 31.

Reactions

—

Submitted

Aug 9, 2026, 10:57 PM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

69.3 tok/s

Prefill

1,755 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

—

RAM

—

OS

Linux

Power

—

Engine

vllm

Engine version

0.26.1rc1.dev457+gc810e5ee9.xpu

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

0.88

Max running seqs

64

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 131072 --gpu-memory-utilization 0.88 --kv-cache-dtype fp8 --max-num-seqs 64 --max-num-batched-tokens 8192 --enable-prefix-caching --language-model-only --speculative-config {"method":"mtp","num_speculative_tokens":4}

Extra flags

--max-model-len 131072 --language-model-only --speculative-config {method:mtp,num_speculative_tokens:4}

Notes

Single-stream C1. vLLM XPU pinned nightly (vllm/vllm-openai-xpu@sha256:2c427ef477da092eb6f2cdbbbd24950b5fa171565b916db69d4c7bb10e68ca97), vllm-xpu-kernels 0.1.12. Model: llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GPTQ-Int4 (dense 27B, GPTQ-INT4 g128 sym, preserved BF16 MTP head). Patches: patch_mtp_nightly.py + patch_mtp_boundary.py. fp8 KV cache (REQUIRED for dense 128K; fp16 does not fit). MTP4 speculative decode, 1-layer draft. gpu-memory-utilization 0.88, scheduler 8192, max-num-seqs 64, prefix cache on with zero hit delta. tokSOut = client post-first rate at p512/g128, median n=5 (not engine-native decode); tokSPrefill = actual input tokens / client TTFT at p2048/g1, median n=5 (not llama-bench pp). 230W configured cap. E2 provisional self-reported; independent reproduction pending. Measured 2026-08-09 Run 31.

Reactions

—

Submitted

Aug 9, 2026, 10:57 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

1,140 tok/s

Prefill

8,715 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

16384

Batch size

64

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

vllm

Engine version

0.21.1.dev18+gec426505f XPU

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

—

Max running seqs

64

Scheduler delay

—

Num parallel

64

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 16384 --max-num-seqs 64 --max-num-batched-tokens 8192 --served-model-name Qwen3.6-35B-A3B-MTP-Preserved-GPTQ-Int4 --language-model-only

Extra flags

--max-model-len 16384 --language-model-only

Notes

CONCURRENCY MAX: 64 concurrent users. Aggregate 1967.7 t/s wall-agg, generation 1139.8 t/s, diverse 512-token prompts, median TPOT 56ms. No-MTP native int4 v4 (MTP+concurrency blocked by GDN kernel). max-num-seqs 64. 165W. Multi-user aggregate throughput, not single-stream decode. 2026-08-07.

Reactions

🔥

Submitted

Aug 7, 2026, 12:19 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

205 tok/s

Prefill

8,715 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

16384

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

vllm

Engine version

0.21.1.dev18+gec426505f XPU

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

—

Max running seqs

64

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 16384 --max-num-seqs 64 --max-num-batched-tokens 8192 --served-model-name Qwen3.6-35B-A3B-MTP-Preserved-GPTQ-Int4 --language-model-only --speculative-config {"method":"mtp","num_speculative_tokens":4}

Extra flags

--max-model-len 16384 --language-model-only --speculative-config {method:mtp,num_speculative_tokens:4}

Notes

Single-stream MTP4 num_spec=4. vLLM 0.21 XPU int4moe + 2 patches: native int4 int8 store, BF16 MTP draft. Recurrent single-layer MTP emits 4 draft tokens/step. short/32 decode 204.6 t/s (grid, prefix-cache-off, warmup discarded). Prefill p4k 8715 t/s @230W. Decode @165W. --max-num-batched-tokens 8192. Model: Qwen3.6-35B-A3B-MTP-Preserved-GPTQ-Int4. 2026-08-07.

Reactions

—

Submitted

Aug 7, 2026, 12:19 PM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

23.0 tok/s

Prefill

1,007 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

1935

Output tokens

128

Prefill tokens

0

Context length

16384

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10255+ (071327508) SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf -ngl 99 -fa on -ctk q8_0 -ctv q4_1 -c 16384 -b 8192 -ub 4096 -t 8 --no-mmap -dev SYCL0

Extra flags

--ctk q8_0 --ctv q4_1 --b 8192 --dev SYCL0

Notes

Single-stream dense 27B. ThinkingCap-Qwen3.6-27B Q4_K_M GGUF. engine rate timings.predicted_per_second. q8_0 K + q4_1 V KV, FA on. 230W (dense scales with power; 79°C peak). vLLM FP8 blocked on this card (no XPU kernel, KeyError PlatformEnum.XPU). Measured 2026-08-06 Run 19.

Reactions

—

Submitted

Aug 6, 2026, 3:05 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

69.0 tok/s

Prefill

1,498 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

1935

Output tokens

128

Prefill tokens

0

Context length

16384

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10255+ (071327508) SYCL

Quantization

Q4_K_XL

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 16384 -b 8192 -ub 4096 -t 8 --no-mmap -dev SYCL0

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192 --dev SYCL0

Notes

Single-stream SYCL llama-server. engine rate timings.predicted_per_second (per AGENTS.md §9.4). Qwen3.6-35B-A3B-UD-Q4_K_XL GGUF. q8_0 K + q4_1 V KV, FA on, -ngl 99 -ncmoe 0. 150W (MoE self-limits to ~140W). MoE baseline for vLLM MTP comparison. Measured 2026-08-06 Run 19.

Reactions

—

Submitted

Aug 6, 2026, 3:05 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

133 tok/s

Prefill

7,535 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

55

Output tokens

32

Prefill tokens

0

Context length

16384

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

vllm

Engine version

0.21.1.dev18+gec426505f XPU

Quantization

GPTQ-Int4

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

auto

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

8192

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

gptq

SGLang quant

—

GPU mem util

0.92

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve /model --quantization gptq --dtype float16 --max-model-len 16384 --gpu-memory-utilization 0.92 --max-num-seqs 1 --max-num-batched-tokens 8192 --language-model-only --speculative-config {"method":"mtp","num_speculative_tokens":1} --cudagraph-capture-sizes 1 2 4 8 16 32

Extra flags

--max-model-len 16384 --language-model-only --speculative-config {method:mtp,num_speculative_tokens:1} --cudagraph-capture-sizes 1

Notes

Single-stream. vLLM 0.21 XPU + 4 in-container patches (github.com/SergiioB/intel-arc-pro-b70-inference-cookbook): native int4 dtype (uint8->int8), BF16 MTP draft, XpuFusedMoe kwarg strip, GDN spec assert->warning. MTP speculative 1 layer num_spec=1. --max-num-batched-tokens 8192 clears chunking cap. Model: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GPTQ-Int4. FLASH_ATTN PIECEWISE graphs. 150W. tokSOut=short/g32 best; tokSPrefill=pp2048 (1945-token prompt). Byte-identical greedy replays verified; KL audit vs eager pending. Measured 2026-08-06 Run 20.

Reactions

—

Submitted

Aug 6, 2026, 3:05 PM

Last edited

—

Model

bottlecapai/ThinkingCap-Qwen3.6-27B

Display name

ThinkingCap-Qwen3.6-27B

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

27.9 tok/s

Prefill

936 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10255+ SYCL (master 071327508)

Quantization

Q6_K

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

draft_mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ThinkingCap-Qwen3.6-27B-Q6_K-MTP.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096 -t 8 --no-mmap -dev SYCL0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192 --dev SYCL0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Notes

ThinkingCap-Qwen3.6-27B Q6_K-MTP with MTP-4 spec decoding (draft-mtp n-max 4). Decode = engine rate timings.predicted_per_second, 4 diverse prompts x 3 reps steady-state (reps 2-3), 200W, 128K ctx (MTP VRAM ceiling for dense). Prefill = llama-bench pp4096 (Run 9, same build). q8_0 K + q4_1 V, FA on.

Reactions

—

Submitted

Aug 5, 2026, 7:59 AM

Last edited

—

Model

bottlecapai/ThinkingCap-Qwen3.6-27B

Display name

ThinkingCap-Qwen3.6-27B

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

26.3 tok/s

Prefill

936 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10255+ SYCL (master 071327508)

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

draft_mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ThinkingCap-Qwen3.6-27B-Q5_K_M-MTP.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096 -t 8 --no-mmap -dev SYCL0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192 --dev SYCL0 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Notes

ThinkingCap-Qwen3.6-27B Q5_K_M-MTP with MTP-4 spec decoding (draft-mtp n-max 4). Decode = engine rate timings.predicted_per_second, 4 diverse prompts x 3 reps steady-state (reps 2-3), 200W, 128K ctx (MTP VRAM ceiling for dense). Prefill = llama-bench pp4096 (Run 9, same build). q8_0 K + q4_1 V, FA on.

Reactions

—

Submitted

Aug 5, 2026, 7:58 AM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

72.6 tok/s

Prefill

2,128 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

32768

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10255+ SYCL (master 071327508)

Quantization

Q4_K_XL

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-bench -m MODEL.gguf -ngl 99 -ncmoe 0 -fa 1 -ctk q8_0 -ctv q4_1 -p 512,4096,8192,32768 -n 128 -r 3 -b 8192 -ub 4096 -t 8 -dev SYCL0

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --p 512,4096,8192,32768 --r 3 --b 8192 --dev SYCL0

Notes

llama-bench tg128 + pp4096 (Run 9). q8_0 K + q4_1 V, FA on, -ngl 99 -ncmoe 0, 150W. Master 0804 build (#25874 quantized-KV XMX FA). Ctx = bench ctx (max prompt 32K).

Reactions

—

Submitted

Aug 5, 2026, 6:49 AM

Last edited

—

Model

bottlecapai/ThinkingCap-Qwen3.6-27B

Display name

ThinkingCap-Qwen3.6-27B

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

27.5 tok/s

Prefill

621 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4014

Output tokens

350

Prefill tokens

0

Context length

204800

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10053+PR25690

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

yes

MTP draft layers

—

Temperature

0.2

Top P

0.95

Top K

20

Min P

0

Repeat penalty

—

Mirostat

—

Command

llama-server -m ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf -dev SYCL0 --parallel 1 -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 204800 -t 8 -b 8192 -ub 4096 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

SYCL/Level Zero; asymmetric KV K=q8_0,V=q4_1; prefill temperature=0; reasoning enabled

Notes

Fresh single-B70 ThinkingCap benchmark on 2026-07-22. Independent phases with prompt cache disabled: 4,014-token prefill at temperature 0 and separate 350-token reasoning/decode task at temperature 0.2. Two warm-ups plus five measured repetitions per phase. Prefill samples=[622.0612,621.6988,621.3770,620.9613,620.5751], CV=0.094%. Decode samples=[26.6528,27.2787,27.2420,28.2763,28.0818], CV=2.421%. MTP-4 decode acceptance=1016/1154 (88.04%). Full GPU offload; Flash Attention on; K=q8_0,V=q4_1; b=8192,ub=4096; 200K context; 165 W. Build: llama.cpp b10053 0dc74e332 plus PR #25690 0bd0ec609. Actual host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04. LocalMaxxing reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw artifact SHA-256: 91eba0e2c34bf90b2d895b7d6e1819c763aadb9ed154dbb0da48b94b368d41d2.

Reactions

—

Submitted

Jul 22, 2026, 8:24 PM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

26.6 tok/s

Prefill

385 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4021

Output tokens

350

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

mtp

Spec model

—

Spec draft model

mtp-gemma-4-31B-it.gguf

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

yes

MTP draft layers

—

Temperature

0.2

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf -dev SYCL0 --parallel 1 -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -t 8 -b 8192 -ub 4096 --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

SYCL/Level Zero; asymmetric KV K=q8_0,V=q4_1; prefill temperature=0

Notes

Single Intel Arc Pro B70 clean-suite result from 2026-07-16. Independent phases: prefill uses the recorded ~4K prompt at temperature 0; decode uses a separate 350-token task at temperature 0.2. Batch/concurrency 1; full GPU offload; Flash Attention on; K=q8_0,V=q4_1; b=8192,ub=4096. Engine: llama.cpp SYCL b9853 (7af4279f4), IntelLLVM 2026.0.0. Actual host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw source SHA-256: 6d4f82df37f469d3f0aba0e1cc3e5642919b1f9c50d2ef5d5c37e4750f20ba4c. Power cap 180 W. MTP-4 enabled; decode draft acceptance 225/297 (75.76%).

Reactions

—

Submitted

Jul 22, 2026, 10:55 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

24.8 tok/s

Prefill

361 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4021

Output tokens

350

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

mtp

Spec model

—

Spec draft model

mtp-gemma-4-31B-it.gguf

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

yes

MTP draft layers

—

Temperature

0.2

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf -dev SYCL0 --parallel 1 -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -t 8 -b 8192 -ub 4096 --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

SYCL/Level Zero; asymmetric KV K=q8_0,V=q4_1; prefill temperature=0

Notes

Single Intel Arc Pro B70 clean-suite result from 2026-07-16. Independent phases: prefill uses the recorded ~4K prompt at temperature 0; decode uses a separate 350-token task at temperature 0.2. Batch/concurrency 1; full GPU offload; Flash Attention on; K=q8_0,V=q4_1; b=8192,ub=4096. Engine: llama.cpp SYCL b9853 (7af4279f4), IntelLLVM 2026.0.0. Actual host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw source SHA-256: 6d4f82df37f469d3f0aba0e1cc3e5642919b1f9c50d2ef5d5c37e4750f20ba4c. Power cap 165 W. MTP-4 enabled; decode draft acceptance 223/298 (74.83%).

Reactions

—

Submitted

Jul 22, 2026, 10:50 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

16.4 tok/s

Prefill

333 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4021

Output tokens

350

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0.2

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf -dev SYCL0 --parallel 1 -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -t 8 -b 8192 -ub 4096

Extra flags

SYCL/Level Zero; asymmetric KV K=q8_0,V=q4_1; prefill temperature=0

Notes

Single Intel Arc Pro B70 clean-suite result from 2026-07-16. Independent phases: prefill uses the recorded ~4K prompt at temperature 0; decode uses a separate 350-token task at temperature 0.2. Batch/concurrency 1; full GPU offload; Flash Attention on; K=q8_0,V=q4_1; b=8192,ub=4096. Engine: llama.cpp SYCL b9853 (7af4279f4), IntelLLVM 2026.0.0. Actual host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw source SHA-256: 6d4f82df37f469d3f0aba0e1cc3e5642919b1f9c50d2ef5d5c37e4750f20ba4c. Power cap 150 W. MTP disabled.

Reactions

—

Submitted

Jul 22, 2026, 10:45 AM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

25.1 tok/s

Prefill

613 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4014

Output tokens

350

Prefill tokens

0

Context length

204800

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

yes

Spec method

mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

yes

MTP draft layers

—

Temperature

0.2

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-MTP-Q5_K_M.gguf -dev SYCL0 --parallel 1 -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 204800 -t 8 -b 8192 -ub 4096 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

SYCL/Level Zero; asymmetric KV K=q8_0,V=q4_1; prefill temperature=0

Notes

Single Intel Arc Pro B70 clean-suite result from 2026-07-16. Independent phases: prefill uses the recorded ~4K prompt at temperature 0; decode uses a separate 350-token task at temperature 0.2. Batch/concurrency 1; full GPU offload; Flash Attention on; K=q8_0,V=q4_1; b=8192,ub=4096. Engine: llama.cpp SYCL b9853 (7af4279f4), IntelLLVM 2026.0.0. Actual host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw source SHA-256: 6d4f82df37f469d3f0aba0e1cc3e5642919b1f9c50d2ef5d5c37e4750f20ba4c. Power cap 165 W. MTP-4 enabled; decode draft acceptance 199/216 (92.13%).

Reactions

—

Submitted

Jul 22, 2026, 10:40 AM

Last edited

—

Model

ornith-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

—

MoE

yes

Output

78.7 tok/s

Prefill

1,590 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4096

Output tokens

128

Prefill tokens

0

Context length

32768

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10053+PR25690

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

1

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-bench -m ornith-1.0-35b-Q5_K_M.gguf -ngl 99 -dev SYCL0 -p 512,4096,8192,32768 -n 128 -r 5 -o json -b 8192 -ub 4096 -mmp 0 -t 8 --flash-attn on -d 0 -ctk q8_0 -ctv q4_1

Extra flags

SYCL/Level Zero; SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1; asymmetric KV: K=q8_0,V=q4_1

Notes

Canonical single-B70 llama-bench result at a verified 150 W power cap. Independent PP4096 and TG128 phases; five measured repetitions per phase. PP4096 samples=[1593.73, 1592.28, 1582.49, 1589.57, 1590.76]; TG128 samples=[78.1363, 78.8193, 78.8413, 78.8379, 78.8436]. PP CV=0.274%; TG CV=0.398%. K cache q8_0, V cache q4_1; full GPU offload; Flash Attention on; no CPU MoE offload; no speculative decoding. Build: llama.cpp b10053 0dc74e332 plus upstream PR #25690 0bd0ec609. Actual benchmark host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04, 150 W cap. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw artifacts checksum-verified.

Reactions

—

Submitted

Jul 22, 2026, 10:27 AM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

67.0 tok/s

Prefill

1,601 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4096

Output tokens

128

Prefill tokens

0

Context length

32768

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10053+PR25690

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

1

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-bench -m Qwen3.6-35B-A3B-UD-Q5_K_M.gguf -ngl 99 -dev SYCL0 -p 512,4096,8192,32768 -n 128 -r 5 -o json -b 8192 -ub 4096 -mmp 0 -t 8 --flash-attn on -d 0 -ctk q8_0 -ctv q4_1

Extra flags

SYCL/Level Zero; SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1; asymmetric KV: K=q8_0,V=q4_1

Notes

Canonical single-B70 llama-bench result at a verified 150 W power cap. Independent PP4096 and TG128 phases; five measured repetitions per phase. PP4096 samples=[1605.62, 1607.22, 1594.85, 1597.68, 1601.33]; TG128 samples=[66.7979, 67.0723, 67.016, 67.0435, 67.0143]. PP CV=0.325%; TG CV=0.163%. K cache q8_0, V cache q4_1; full GPU offload; Flash Attention on; no CPU MoE offload; no speculative decoding. Build: llama.cpp b10053 0dc74e332 plus upstream PR #25690 0bd0ec609. Actual benchmark host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04, 150 W cap. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw artifacts checksum-verified.

Reactions

—

Submitted

Jul 22, 2026, 10:21 AM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

3B

MoE

yes

Output

69.7 tok/s

Prefill

1,603 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4096

Output tokens

128

Prefill tokens

0

Context length

32768

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b10053+PR25690

Quantization

Q4_K_XL

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

1

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -dev SYCL0 -p 512,4096,8192,32768 -n 128 -r 5 -o json -b 8192 -ub 4096 -mmp 0 -t 8 --flash-attn on -d 0 -ctk q8_0 -ctv q4_1

Extra flags

SYCL/Level Zero; SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1; asymmetric KV: K=q8_0,V=q4_1

Notes

Canonical single-B70 llama-bench result at a verified 150 W power cap. Independent PP4096 and TG128 phases; five measured repetitions per phase. PP4096 samples=[1605.32, 1603.1, 1595.12, 1604.54, 1609.38]; TG128 samples=[69.5063, 69.7579, 69.7894, 69.7738, 69.643]. PP CV=0.326%; TG CV=0.172%. K cache q8_0, V cache q4_1; full GPU offload; Flash Attention on; no CPU MoE offload; no speculative decoding. Build: llama.cpp b10053 0dc74e332 plus upstream PR #25690 0bd0ec609. Actual benchmark host: AMD Ryzen 7 5700X3D, 32 GB RAM, Ubuntu 26.04, 150 W cap. LocalMaxxing currently reuses an older B70 hardware profile for this account; its structured CPU/RAM/OS/power fields are stale and should be ignored. Raw artifacts checksum-verified.

Reactions

—

Submitted

Jul 22, 2026, 10:15 AM

Last edited

Jul 22, 2026, 10:24 AM

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

26.6 tok/s

Prefill

385 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M + MTP-4 @180W best dense decode in sweep. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

24.8 tok/s

Prefill

361 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M + Unsloth mtp-gemma-4-31B-it draft-mtp n-max 4 @165W. Decode +51% vs base. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

31B

Active params

—

MoE

no

Output

16.4 tok/s

Prefill

333 tok/s

Total

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M BASE no MTP @150W. Single-stream dense control. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—