Sandberg

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

47.0 tok/s

Prefill

1,568 tok/s

Total

341 tok/s

TTFT

2,613 ms

Peak VRAM

11.2GB

Power draw

168.3W

Hardware cost

—

Prompt tokens

4096

Output tokens

512

Prefill tokens

4096

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4096 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4096. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 46.9-47.0 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:52 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

51.1 tok/s

Prefill

1,611 tok/s

Total

226 tok/s

TTFT

1,271 ms

Peak VRAM

11.2GB

Power draw

168.2W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 51.0-51.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:52 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

44.3 tok/s

Prefill

7,411 tok/s

Total

64.1 tok/s

TTFT

31.2 ms

Peak VRAM

22.2GB

Power draw

348.2W

Hardware cost

—

Prompt tokens

231

Output tokens

512

Prefill tokens

0

Context length

106496

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 9 7900X (12C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

vllm

Engine version

0.32.0

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

int4_per_token_head

KV cache size

—

Prefix caching

yes

Attention backend

TRITON_ATTN

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 106496 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code --kv-cache-dtype int4_per_token_head --attention-backend TRITON_ATTN

Extra flags

--max-model-len 106496

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: int4_per_token_head. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 350 W limit (VBIOS max), PCIe Gen4 x16 | AMD Ryzen 9 7900X 12C | 32 GB DDR5 | Ubuntu 24.04.5, NVIDIA driver 595.99.02. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 44.3-44.3 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:51 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

44.2 tok/s

Prefill

9,202 tok/s

Total

69.0 tok/s

TTFT

31.4 ms

Peak VRAM

22.2GB

Power draw

348.2W

Hardware cost

—

Prompt tokens

289

Output tokens

512

Prefill tokens

0

Context length

106496

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 9 7900X (12C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

vllm

Engine version

0.32.0

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

int4_per_token_head

KV cache size

—

Prefix caching

yes

Attention backend

TRITON_ATTN

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 106496 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code --kv-cache-dtype int4_per_token_head --attention-backend TRITON_ATTN

Extra flags

--max-model-len 106496

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: int4_per_token_head. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 350 W limit (VBIOS max), PCIe Gen4 x16 | AMD Ryzen 9 7900X 12C | 32 GB DDR5 | Ubuntu 24.04.5, NVIDIA driver 595.99.02. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 44.2-44.3 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:51 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

47.8 tok/s

Prefill

3,395 tok/s

Total

68.8 tok/s

TTFT

67.7 ms

Peak VRAM

22.3GB

Power draw

416.3W

Hardware cost

—

Prompt tokens

230

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 47.8-47.8 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:49 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

47.7 tok/s

Prefill

3,414 tok/s

Total

74.1 tok/s

TTFT

84.6 ms

Peak VRAM

22.3GB

Power draw

416.3W

Hardware cost

—

Prompt tokens

289

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 47.7-47.8 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:49 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

21.4 tok/s

Prefill

1,391 tok/s

Total

1,112 tok/s

TTFT

93,471 ms

Peak VRAM

19.9GB

Power draw

415.7W

Hardware cost

—

Prompt tokens

130035

Output tokens

512

Prefill tokens

130035

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 130035 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130035. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 21.4-21.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:47 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

48.2 tok/s

Prefill

2,919 tok/s

Total

1,523 tok/s

TTFT

11,223 ms

Peak VRAM

19.9GB

Power draw

411.4W

Hardware cost

—

Prompt tokens

32765

Output tokens

512

Prefill tokens

32765

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32765 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32765. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 48.2-48.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:47 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

60.7 tok/s

Prefill

3,435 tok/s

Total

1,279 tok/s

TTFT

4,770 ms

Peak VRAM

19.9GB

Power draw

417.9W

Hardware cost

—

Prompt tokens

16382

Output tokens

512

Prefill tokens

16382

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16382 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16382. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 60.7-60.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:47 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

69.3 tok/s

Prefill

3,852 tok/s

Total

915 tok/s

TTFT

2,127 ms

Peak VRAM

19.9GB

Power draw

411.5W

Hardware cost

—

Prompt tokens

8192

Output tokens

512

Prefill tokens

8192

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8192 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8192. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 69.3-69.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:46 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

74.7 tok/s

Prefill

4,081 tok/s

Total

587 tok/s

TTFT

1,004 ms

Peak VRAM

19.9GB

Power draw

415.3W

Hardware cost

—

Prompt tokens

4096

Output tokens

512

Prefill tokens

4096

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4096 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4096. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 74.7-74.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:46 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

77.7 tok/s

Prefill

4,135 tok/s

Total

362 tok/s

TTFT

495 ms

Peak VRAM

19.9GB

Power draw

414.5W

Hardware cost

—

Prompt tokens

2048

Output tokens

512

Prefill tokens

2048

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2048 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2048. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 77.7-77.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:46 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

80.5 tok/s

Prefill

2,844 tok/s

Total

115 tok/s

TTFT

80.9 ms

Peak VRAM

19.9GB

Power draw

415.4W

Hardware cost

—

Prompt tokens

230

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 80.5-80.5 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:37 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

80.5 tok/s

Prefill

2,962 tok/s

Total

124 tok/s

TTFT

97.6 ms

Peak VRAM

19.9GB

Power draw

415.4W

Hardware cost

—

Prompt tokens

289

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 80.4-80.5 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:37 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

23.2 tok/s

Prefill

1,364 tok/s

Total

1,112 tok/s

TTFT

95,338 ms

Peak VRAM

16.4GB

Power draw

416.3W

Hardware cost

—

Prompt tokens

130034

Output tokens

512

Prefill tokens

130034

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 130034 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130034. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 23.2-23.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:36 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

59.7 tok/s

Prefill

2,683 tok/s

Total

1,600 tok/s

TTFT

12,213 ms

Peak VRAM

16.4GB

Power draw

409.6W

Hardware cost

—

Prompt tokens

32765

Output tokens

512

Prefill tokens

32765

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32765 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32765. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 59.6-59.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:35 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

80.7 tok/s

Prefill

3,285 tok/s

Total

1,491 tok/s

TTFT

4,988 ms

Peak VRAM

16.4GB

Power draw

412.3W

Hardware cost

—

Prompt tokens

16382

Output tokens

512

Prefill tokens

16382

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16382 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16382. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 80.7-80.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:35 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

97.4 tok/s

Prefill

3,674 tok/s

Total

1,162 tok/s

TTFT

2,229 ms

Peak VRAM

16.4GB

Power draw

413.2W

Hardware cost

—

Prompt tokens

8191

Output tokens

512

Prefill tokens

8191

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8191 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8191. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 97.2-97.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:35 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

109 tok/s

Prefill

3,882 tok/s

Total

799 tok/s

TTFT

1,055 ms

Peak VRAM

16.4GB

Power draw

412.9W

Hardware cost

—

Prompt tokens

4096

Output tokens

512

Prefill tokens

4096

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4096 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4096. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 108.5-108.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:35 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

115 tok/s

Prefill

3,917 tok/s

Total

516 tok/s

TTFT

523 ms

Peak VRAM

16.4GB

Power draw

413.4W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 115.3-115.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 1:35 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

55.1 tok/s

Prefill

1,220 tok/s

Total

78.3 tok/s

TTFT

189 ms

Peak VRAM

11.2GB

Power draw

168.8W

Hardware cost

—

Prompt tokens

231

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 55.0-55.2 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:28 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

55.2 tok/s

Prefill

1,323 tok/s

Total

84.5 tok/s

TTFT

220 ms

Peak VRAM

11.2GB

Power draw

168.8W

Hardware cost

—

Prompt tokens

291

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 55.1-55.2 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:28 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

122 tok/s

Prefill

2,729 tok/s

Total

173 tok/s

TTFT

84.6 ms

Peak VRAM

16.4GB

Power draw

415.2W

Hardware cost

—

Prompt tokens

231

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q8_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -ctk q8_0 -ctv q8_0 -np 1 -fa on -fit off

Extra flags

--ctk q8_0 --ctv q8_0 --fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q8_0 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: q8_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 121.7-122.0 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 1:27 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

122 tok/s

Prefill

2,882 tok/s

Total

186 tok/s

TTFT

101 ms

Peak VRAM

16.4GB

Power draw

415.2W