Sandberg

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

62.5 tok/s

Prefill

7,151 tok/s

Total

525 tok/s

TTFT

573 ms

Peak VRAM

59.3GB

Power draw

180.8W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

vllm

Engine version

0.32.0rc1.dev249+g18305948a

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

auto

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code

Extra flags

--max-model-len 131072

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 62.4-62.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:07 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

63.7 tok/s

Prefill

7,114 tok/s

Total

308 tok/s

TTFT

288 ms

Peak VRAM

59.3GB

Power draw

177.9W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

vllm

Engine version

0.32.0rc1.dev249+g18305948a

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

auto

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code

Extra flags

--max-model-len 131072

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 63.6-63.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:07 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

65.3 tok/s

Prefill

5,250 tok/s

Total

94.2 tok/s

TTFT

44.0 ms

Peak VRAM

59.3GB

Power draw

177.9W

Hardware cost

—

Prompt tokens

231

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

vllm

Engine version

0.32.0rc1.dev249+g18305948a

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

auto

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code

Extra flags

--max-model-len 131072

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 65.3-65.3 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 4:00 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

65.3 tok/s

Prefill

5,335 tok/s

Total

101 tok/s

TTFT

54.2 ms

Peak VRAM

59.3GB

Power draw

177.9W

Hardware cost

—

Prompt tokens

289

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

vllm

Engine version

0.32.0rc1.dev249+g18305948a

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

auto

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 131072 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code

Extra flags

--max-model-len 131072

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 65.2-65.3 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 3:59 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

30.6 tok/s

Prefill

1,295 tok/s

Total

1,115 tok/s

TTFT

100,404 ms

Peak VRAM

36.2GB

Power draw

170.2W

Hardware cost

—

Prompt tokens

130034

Output tokens

512

Prefill tokens

130034

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 130034 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130034. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 30.6-30.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

49.1 tok/s

Prefill

2,337 tok/s

Total

1,361 tok/s

TTFT

14,022 ms

Peak VRAM

36.2GB

Power draw

168.7W

Hardware cost

—

Prompt tokens

32765

Output tokens

512

Prefill tokens

32765

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32765 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32765. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 49.1-49.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

54.6 tok/s

Prefill

2,184 tok/s

Total

1,001 tok/s

TTFT

7,502 ms

Peak VRAM

36.2GB

Power draw

169.1W

Hardware cost

—

Prompt tokens

16382

Output tokens

512

Prefill tokens

16382

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16382 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16382. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 54.5-54.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

58.0 tok/s

Prefill

3,080 tok/s

Total

758 tok/s

TTFT

2,660 ms

Peak VRAM

36.2GB

Power draw

172.6W

Hardware cost

—

Prompt tokens

8192

Output tokens

512

Prefill tokens

8192

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8192 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8192. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 57.9-58.0 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

60.1 tok/s

Prefill

2,416 tok/s

Total

451 tok/s

TTFT

1,696 ms

Peak VRAM

36.2GB

Power draw

171.7W

Hardware cost

—

Prompt tokens

4096

Output tokens

512

Prefill tokens

4096

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4096 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4096. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 60.1-60.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

61.1 tok/s

Prefill

2,306 tok/s

Total

276 tok/s

TTFT

888 ms

Peak VRAM

36.2GB

Power draw

181.2W

Hardware cost

—

Prompt tokens

2048

Output tokens

512

Prefill tokens

2048

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2048 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2048. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 61.1-61.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:41 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

62.4 tok/s

Prefill

1,116 tok/s

Total

88.2 tok/s

TTFT

206 ms

Peak VRAM

36.2GB

Power draw

180.5W

Hardware cost

—

Prompt tokens

230

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 62.3-62.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 3:30 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

62.5 tok/s

Prefill

1,258 tok/s

Total

95.2 tok/s

TTFT

230 ms

Peak VRAM

36.2GB

Power draw

180.5W

Hardware cost

—

Prompt tokens

289

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

F16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-F16-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-F16-verbose.gguf (F16). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 62.5-62.6 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 3:30 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

34.9 tok/s

Prefill

1,086 tok/s

Total

971 tok/s

TTFT

119,790 ms

Peak VRAM

28.9GB

Power draw

170.5W

Hardware cost

—

Prompt tokens

130033

Output tokens

512

Prefill tokens

130033

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 130033 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130033. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 34.9-35.0 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:27 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

62.2 tok/s

Prefill

1,455 tok/s

Total

1,082 tok/s

TTFT

22,520 ms

Peak VRAM

28.9GB

Power draw

161.3W

Hardware cost

—

Prompt tokens

32765

Output tokens

512

Prefill tokens

32765

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32765 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32765. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 62.2-62.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:27 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

71.4 tok/s

Prefill

1,626 tok/s

Total

980 tok/s

TTFT

10,073 ms

Peak VRAM

28.9GB

Power draw

163.5W

Hardware cost

—

Prompt tokens

16381

Output tokens

512

Prefill tokens

16381

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16381 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16381. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 68.7-71.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:27 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

77.0 tok/s

Prefill

2,049 tok/s

Total

818 tok/s

TTFT

3,998 ms

Peak VRAM

28.9GB

Power draw

168.5W

Hardware cost

—

Prompt tokens

8191

Output tokens

512

Prefill tokens

8191

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8191 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8191. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 76.9-77.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:06 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

80.4 tok/s

Prefill

1,717 tok/s

Total

526 tok/s

TTFT

2,385 ms

Peak VRAM

28.9GB

Power draw

171.3W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 80.3-80.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:06 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

82.4 tok/s

Prefill

1,673 tok/s

Total

344 tok/s

TTFT

1,225 ms

Peak VRAM

28.9GB

Power draw

171.8W

Hardware cost

—

Prompt tokens

2048

Output tokens

512

Prefill tokens

2048

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2048 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2048. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 82.1-82.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 3:06 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

84.8 tok/s

Prefill

913 tok/s

Total

118 tok/s

TTFT

252 ms

Peak VRAM

28.9GB

Power draw

172.9W

Hardware cost

—

Prompt tokens

230

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 84.7-85.0 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 2:55 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

85.4 tok/s

Prefill

958 tok/s

Total

127 tok/s

TTFT

303 ms

Peak VRAM

28.9GB

Power draw

172.9W

Hardware cost

—

Prompt tokens

290

Output tokens

512

Prefill tokens

0

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q8_0-verbose.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q8_0-verbose.gguf (Q8_0). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), verbose variant: webAI publishes Q8_0 and F16 GGUFs only for verbose; same architecture as balanced. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 85.1-85.8 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 2:55 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

37.3 tok/s

Prefill

1,081 tok/s

Total

974 tok/s

TTFT

120,270 ms

Peak VRAM

25.3GB

Power draw

166.7W

Hardware cost

—

Prompt tokens

130035

Output tokens

512

Prefill tokens

130035

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 130035 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130035. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 37.1-37.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 2:53 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

16.3 tok/s

Prefill

501 tok/s

Total

438 tok/s

TTFT

210,531 ms

Peak VRAM

22.2GB

Power draw

339.8W

Hardware cost

—

Prompt tokens

105461

Output tokens

512

Prefill tokens

105461

Context length

106496

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen 9 7900X (12C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

vllm

Engine version

0.32.0

Quantization

BF16

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

int4_per_token_head

KV cache size

—

Prefix caching

yes

Attention backend

TRITON_ATTN

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.94

Max running seqs

128

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

vllm serve webAI-Official/TwIL-8B@58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630 --max-model-len 106496 --gpu-memory-utilization 0.94 --max-num-seqs 128 --enable-prefix-caching --trust-remote-code --kv-cache-dtype int4_per_token_head --attention-backend TRITON_ATTN

Extra flags

--max-model-len 106496

Notes

Untuned out-of-the-box baseline (same server config as our LMX matrix for this model). Spec decode: none. KV cache: int4_per_token_head. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 350 W limit (VBIOS max), PCIe Gen4 x16 | AMD Ryzen 9 7900X 12C | 32 GB DDR5 | Ubuntu 24.04.5, NVIDIA driver 595.99.02. Depth run: 105461 tokens of context (103k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 105461. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 16.3-16.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 2:53 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

70.4 tok/s

Prefill

1,456 tok/s

Total

1,118 tok/s

TTFT

22,504 ms

Peak VRAM

25.3GB

Power draw

160.3W

Hardware cost

—

Prompt tokens

32766

Output tokens

512

Prefill tokens

32766

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m TwIL-8B-Q4_K_M-balanced.gguf -ngl 999 -c 131072 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: TwIL-8B-Q4_K_M-balanced.gguf (Q4_K_M). TwIL-8B by webAI (Granite-4.2-8B post-trained with SFT + GRPO/DAPO), balanced variant (the default). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 131072 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32766 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32766. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 69.8-71.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 2:51 PM

Last edited

—

Model

webAI-Official/TwIL-8B

Display name

TwIL-8B

Base model

granite-4.2-8b

Revision

58d49f0d9909e62f9c2fb09e99fc5c76f3f2d630

Family

—

Parameters

8B

Active params

—

MoE

no

Output

28.9 tok/s

Prefill

1,361 tok/s

Total

796 tok/s

TTFT

24,079 ms

Peak VRAM

22.2GB

Power draw

342.6W