SergiioB

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

33B

Active params

—

MoE

no

Output tok/s

26.6

Prefill tok/s

384.8

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M + MTP-4 @180W best dense decode in sweep. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

33B

Active params

—

MoE

no

Output tok/s

24.8

Prefill tok/s

361.2

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 4 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M + Unsloth mtp-gemma-4-31B-it draft-mtp n-max 4 @165W. Decode +51% vs base. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

google/gemma-4-31B-it

Display name

gemma-4-31B-it

Base model

gemma-4-31B

Revision

main

Family

Gemma

Parameters

33B

Active params

—

MoE

no

Output tok/s

16.4

Prefill tok/s

332.9

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-31B-it-Q4_K_M.gguf -fa on -ctk q8_0 -ctv q4_1 -c 131072 -b 8192 -ub 4096

Extra flags

--ctk q8_0 --ctv q4_1 --b 8192

Notes

Gemma-4-31B-it Q4_K_M BASE no MTP @150W. Single-stream dense control. Prefill ~4k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

Qwen/Qwen3.5-27B

Display name

Qwen3.5-27B

Base model

—

Revision

main

Family

Qwen

Parameters

28B

Active params

—

MoE

no

Output tok/s

25.1

Prefill tok/s

613.0

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

204800

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-MTP-Q5_K_M.gguf --spec-type draft-mtp --spec-draft-n-max 4 -fa on -ctk q8_0 -ctv q4_1 -c 204800 -b 8192 -ub 4096

Extra flags

--spec-type draft-mtp --spec-draft-n-max 4 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Qwen3.6-27B-MTP-Q5_K_M dense + draft-mtp n-max 4. Single-stream. Prefill ~4k. Profile qwen27-mtp-q5-256k-165w @165W. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

Qwen/Qwen3.5-35B-A3B

Display name

Qwen3.5-35B-A3B

Base model

Qwen3.5-35B-A3B-Base

Revision

main

Family

Qwen

Parameters

35B

Active params

—

MoE

no

Output tok/s

62.8

Prefill tok/s

1682.4

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q4_K_XL

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 262144 -b 8192 -ub 4096

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Qwen3.6-35B-A3B UD-Q4_K_XL. Single-stream. Prefill ~4k. Profile qwen35-q4-256k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

Qwen/Qwen3.5-35B-A3B

Display name

Qwen3.5-35B-A3B

Base model

Qwen3.5-35B-A3B-Base

Revision

main

Family

Qwen

Parameters

35B

Active params

—

MoE

no

Output tok/s

61.5

Prefill tok/s

1690.2

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-35B-A3B-UD-Q5_K_M.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 262144 -b 8192 -ub 4096

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192

Notes

Qwen3.6-35B-A3B UD-Q5_K_M GGUF served as active. Single-stream. Prefill ~4k tokens. Profile qwen35-q5-256k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

69.3

Prefill tok/s

1725.9

Total tok/s

—

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q8_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q5_K_M.gguf -ngl 99 -ncmoe 0 -fa on -ctk q8_0 -ctv q4_1 -c 262144 -b 8192 -ub 4096 -dev SYCL0

Extra flags

--ncmoe 0 --ctk q8_0 --ctv q4_1 --b 8192 --dev SYCL0

Notes

Single-stream SYCL llama-server. Decode from generation task; prefill from ~4k prompt timings.prompt_per_second. -b8192 -ub4096 FA on ctk q8_0 ctv q4_1. Profile ornith35-q5-256k. 2026-07-16.

Reactions

—

Submitted

Jul 16, 2026, 9:49 AM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

67.6

Prefill tok/s

1242.3

Total tok/s

67.6

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q5_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

1

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q5_K_M.gguf --parallel 1 --ctx-size 131072 -fa on -ctk q5_0 -ctv q4_1 -b 8192 -ub 4096

Extra flags

--ctk q5_0 --ctv q4_1 --b 8192

Notes

PEAK SINGLE-STREAM PREFILL TEST: Measured using a contiguous 3000-token prompt block without concurrency (batch=1). Uncovers the raw 1,242 tok/s prefill capacity of the B70 without the massive context-switching overhead incurred in fleet concurrency. Hardware capped at optimal 165W/2400MHz.

Reactions

—

Submitted

Jul 6, 2026, 9:41 AM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

164.5

Prefill tok/s

383.4

Total tok/s

171.1

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

32

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q5_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

32

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q5_K_M.gguf --parallel 32 --ctx-size 131072 -fa on -ctk q5_0 -ctv q4_1 -b 8192 -ub 4096

Extra flags

--ctk q5_0 --ctv q4_1 --b 8192

Notes

POWER/FREQUENCY UNLOCKED TEST: max_freq pushed to 2800 MHz, power1_cap relaxed to 230W. Telemetry logged 196W sustained draw at 73C. Memory bandwidth bottleneck confirmed: +33% power yielded only +8% throughput over the 165W/2400MHz baseline.

Reactions

—

Submitted

Jul 6, 2026, 9:33 AM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

153.0

Prefill tok/s

500.2

Total tok/s

145.9

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

32

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q5_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

32

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q5_K_M.gguf --parallel 32 --ctx-size 131072 -fa on -ctk q5_0 -ctv q4_1 -b 8192 -ub 4096

Extra flags

--ctk q5_0 --ctv q4_1 --b 8192

Notes

UPDATED: Extracted pure decode/prefill metrics from server logs. GGML_SYCL_DISABLE_DNN=1, -b 8192 -ub 4096. 32 concurrent clients. VRAM: 70C, GPU: 69C, Power: 149.5W.

Reactions

—

Submitted

Jul 6, 2026, 9:22 AM

Last edited

—

Model

Frosty40/Ornith-1.0-35B-B70-Turbo

Display name

Ornith-1.0-35B-B70-Turbo

Base model

Ornith-1.0-35B

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

145.9

Prefill tok/s

—

Total tok/s

145.9

TTFT

—

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

0

Prefill tokens

—

Context length

131072

Batch size

32

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9853 SYCL

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

q5_0/q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

4096

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

32

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q5_K_M.gguf --parallel 32 --ctx-size 131072 -fa on -ctk q5_0 -ctv q4_1 -b 8192 -ub 4096

Extra flags

--ctk q5_0 --ctv q4_1 --b 8192

Notes

GGML_SYCL_DISABLE_DNN=1, -b 8192 -ub 4096. 32 concurrent clients. Aggregate throughput for fleet/concurrency usage.

Reactions

—

Submitted

Jul 6, 2026, 9:04 AM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-9B

Display name

Ornith-1.0-9B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

no

Output tok/s

50.3

Prefill tok/s

553.9

Total tok/s

56.2

TTFT

57.8ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q6_K

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

flash_attn

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-9b-Q6_K.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV. Q6_K weights. 150W power cap. 256K context at 50 t/s — only 15GB VRAM used, could reach 512K+. Highest context headroom of any tested model. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 64C.

Reactions

—

Submitted

Jul 1, 2026, 8:42 PM

Last edited

—

Model

InternScience/Agents-A1

Display name

Agents-A1

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

—

MoE

yes

Output tok/s

73.2

Prefill tok/s

416.1

Total tok/s

80.9

TTFT

76.9ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

flash_attn

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Agents-A1-Q4_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV (92.65% tail precision). 150W power cap. 256K context at 73 t/s engine. Agent-optimized 35B MoE model. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 67C. Zero context scaling penalty.

Reactions

—

Submitted

Jul 1, 2026, 8:41 PM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

74.3

Prefill tok/s

440.8

Total tok/s

82.1

TTFT

72.6ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

flash_attn

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q4_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV (92.65% tail precision). 150W power cap — MoE compute-bound. 256K context at 74 t/s engine — fastest overall. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 67C. Zero context scaling penalty vs 128K.

Reactions

—

Submitted

Jul 1, 2026, 8:40 PM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

28B

Active params

—

MoE

no

Output tok/s

29.7

Prefill tok/s

102.4

Total tok/s

32.5

TTFT

312.4ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

flash_attn

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

MTP enabled

yes

MTP draft layers

4

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-MTP-Q5_K_M.gguf -c 131072 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Notes

q5_0-q4_1 asymmetric KV. 230W stock power cap — max perf config. MTP-4 speculative decoding (95% draft acceptance). Dense 27B is memory-bandwidth bound, benefits from max power. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 77C under load.

Reactions

—

Submitted

Jul 1, 2026, 8:39 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

36B

Active params

3B

MoE

yes

Output tok/s

67

Prefill tok/s

464.9

Total tok/s

74

TTFT

68.8ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

flash_attn

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-35B-A3B-UD-Q5_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV (92.65% tail precision, 38% VRAM savings vs q8_0). 150W power cap — MoE compute-bound, higher power provides negligible benefit. 256K context at 62 t/s engine. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 65C.

Reactions

—

Submitted

Jul 1, 2026, 8:38 PM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-9B

Display name

Ornith-1.0-9B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

no

Output tok/s

50.3

Prefill tok/s

553.9

Total tok/s

—

TTFT

57.8ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q6_K

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-9b-Q6_K.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV cache. Q6_K model weights. 150W power cap. 256K context — only 15GB VRAM used, could reach 512K+. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 64C under load. Highest context headroom of any tested model.

Reactions

—

Submitted

Jul 1, 2026, 8:34 PM

Last edited

—

Model

InternScience/Agents-A1

Display name

Agents-A1

Base model

—

Revision

main

Family

Qwen

Parameters

35B

Active params

—

MoE

yes

Output tok/s

73.2

Prefill tok/s

416.1

Total tok/s

—

TTFT

76.9ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Agents-A1-Q4_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV cache (92.65% tail precision, 38% less VRAM than q8_0). 150W power cap — MoE model, compute-bound. 256K context with q5_0-q4_1 KV. Agent-optimized model at massive context window. Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 67C under load. Zero context scaling penalty vs 128K.

Reactions

—

Submitted

Jul 1, 2026, 8:33 PM

Last edited

—

Model

deepreinforce-ai/Ornith-1.0-35B

Display name

Ornith-1.0-35B

Base model

—

Revision

main

Family

Qwen

Parameters

0B

Active params

—

MoE

yes

Output tok/s

74.3

Prefill tok/s

440.8

Total tok/s

—

TTFT

72.6ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q4_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m ornith-1.0-35b-Q4_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV cache (92.65% tail precision, 38% less VRAM than q8_0). 150W power cap — MoE model is compute-bound, higher power provides negligible benefit. 256K context at 150W with q5_0-q4_1 KV (would need q8_0-q8_0 for same context, costing +38% VRAM). Intel Arc Pro B70 32GB. llama.cpp SYCL b9851. oneAPI 2026.0.0. GPU temp: 67C under load. Zero context scaling penalty vs 128K.

Reactions

—

Submitted

Jul 1, 2026, 8:32 PM

Last edited

—

Model

Qwen/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

—

Revision

main

Family

Qwen

Parameters

28B

Active params

—

MoE

no

Output tok/s

26.6

Prefill tok/s

103.2

Total tok/s

—

TTFT

310.0ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

204800

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

4

Spec ngram

—

Spec draft TP

—

MTP enabled

yes

MTP draft layers

4

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-MTP-Q5_K_M.gguf -c 204800 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1 --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.75

Notes

q5_0-q4_1 asymmetric KV cache. Intel oneAPI 2026.0.0 + IntelLLVM 2026.0.0. SYCL backend.

Reactions

—

Submitted

Jul 1, 2026, 8:30 PM

Last edited

—

Model

Qwen/Qwen3.6-35B-A3B

Display name

Qwen3.6-35B-A3B

Base model

—

Revision

main

Family

Qwen

Parameters

36B

Active params

3B

MoE

yes

Output tok/s

67

Prefill tok/s

464.9

Total tok/s

—

TTFT

68.8ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

32

Output tokens

256

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

Intel Arc Pro B70 32GB

GPU slots

—

GPU count

1

VRAM

32GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 7 7840HS w/ Radeon 780M Graphics

RAM

25.8GB

OS

Windows 11 Pro 10.0.26100 build 26100

Power

230W

Engine

llama.cpp

Engine version

b9851 (0eca4d490)

Quantization

Q5_K_M

Backend

xpu

Tensor parallel

—

Pipeline parallel

—

GPU layers

99

Split mode

—

KV cache dtype

q5_0-q4_1

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

MTP enabled

no

MTP draft layers

—

Temperature

8

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-35B-A3B-UD-Q5_K_M.gguf -c 262144 -ngl 99 -ncmoe 0 -fa on -ctk q5_0 -ctv q4_1 -t 8 --no-mmap

Extra flags

--ncmoe 0 --ctk q5_0 --ctv q4_1

Notes

q5_0-q4_1 asymmetric KV cache. Intel oneAPI 2026.0.0 + IntelLLVM 2026.0.0. SYCL backend.

Reactions

—

Submitted

Jul 1, 2026, 8:28 PM

Last edited

—