Sandberg

Model

XHToken/Spark-X2.6-4B-GGUF

Display name

Spark-X2.6-4B-GGUF

Base model

Spark-X2.6-4B

Revision

3c33ab38746828196b9b5979c739ee2d0bef894d

Family

—

Parameters

4B

Active params

—

MoE

no

Output

114 tok/s

Prefill

8,132 tok/s

Total

1,581 tok/s

TTFT

1,007 ms

Peak VRAM

19.6GB

Power draw

406.7W

Hardware cost

—

Prompt tokens

8191

Output tokens

512

Prefill tokens

8191

Context length

1048576

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Spark-X2.6-4B-Q8_0.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Spark-X2.6-4B-Q8_0.gguf (Q8_0). Spark-X2.6-4B by XHToken (official GGUF). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8191 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8191. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 113.8-113.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 5:32 AM

Last edited

—

Model

XHToken/Spark-X2.6-4B-GGUF

Display name

Spark-X2.6-4B-GGUF

Base model

Spark-X2.6-4B

Revision

3c33ab38746828196b9b5979c739ee2d0bef894d

Family

—

Parameters

4B

Active params

—

MoE

no

Output

116 tok/s

Prefill

7,879 tok/s

Total

936 tok/s

TTFT

520 ms

Peak VRAM

19.6GB

Power draw

398.7W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

1048576

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Spark-X2.6-4B-Q8_0.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Spark-X2.6-4B-Q8_0.gguf (Q8_0). Spark-X2.6-4B by XHToken (official GGUF). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 116.3-116.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 5:32 AM

Last edited

—

Model

XHToken/Spark-X2.6-4B-GGUF

Display name

Spark-X2.6-4B-GGUF

Base model

Spark-X2.6-4B

Revision

3c33ab38746828196b9b5979c739ee2d0bef894d

Family

—

Parameters

4B

Active params

—

MoE

no

Output

118 tok/s

Prefill

7,064 tok/s

Total

554 tok/s

TTFT

290 ms

Peak VRAM

19.6GB

Power draw

394.7W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

1048576

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q8_0

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Spark-X2.6-4B-Q8_0.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Spark-X2.6-4B-Q8_0.gguf (Q8_0). Spark-X2.6-4B by XHToken (official GGUF). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 118.2-118.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 5:32 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

126 tok/s

Prefill

7,067 tok/s

Total

6,380 tok/s

TTFT

36,944 ms

Peak VRAM

4.6GB

Power draw

388W

Hardware cost

—

Prompt tokens

261095

Output tokens

512

Prefill tokens

261095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 261095 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 126.1-126.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

238 tok/s

Prefill

16,005 tok/s

Total

7,929 tok/s

TTFT

2,047 ms

Peak VRAM

4.6GB

Power draw

352W

Hardware cost

—

Prompt tokens

32763

Output tokens

512

Prefill tokens

32763

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32763 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32763. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 237.9-239.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

256 tok/s

Prefill

18,038 tok/s

Total

5,801 tok/s

TTFT

908 ms

Peak VRAM

4.6GB

Power draw

342.9W

Hardware cost

—

Prompt tokens

16381

Output tokens

512

Prefill tokens

16381

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16381 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16381. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 255.2-256.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

264 tok/s

Prefill

18,370 tok/s

Total

3,650 tok/s

TTFT

446 ms

Peak VRAM

4.6GB

Power draw

328.2W

Hardware cost

—

Prompt tokens

8191

Output tokens

512

Prefill tokens

8191

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8191 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8191. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 264.1-265.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

269 tok/s

Prefill

17,623 tok/s

Total

2,153 tok/s

TTFT

232 ms

Peak VRAM

4.6GB

Power draw

315.5W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 268.0-269.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

271 tok/s

Prefill

15,279 tok/s

Total

1,265 tok/s

TTFT

134 ms

Peak VRAM

4.6GB

Power draw

307.7W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 270.7-271.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:28 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

273 tok/s

Prefill

6,824 tok/s

Total

382 tok/s

TTFT

31.8 ms

Peak VRAM

4.6GB

Power draw

309.9W

Hardware cost

—

Prompt tokens

217

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 272.8-273.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 4, 2026, 4:25 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

275 tok/s

Prefill

6,456 tok/s

Total

415 tok/s

TTFT

42.9 ms

Peak VRAM

4.6GB

Power draw

309.9W

Hardware cost

—

Prompt tokens

277

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 274.6-275.9 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 4, 2026, 4:25 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

118 tok/s

Prefill

6,951 tok/s

Total

6,245 tok/s

TTFT

37,565 ms

Peak VRAM

5.1GB

Power draw

391.2W

Hardware cost

—

Prompt tokens

261095

Output tokens

512

Prefill tokens

261095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 261095 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 118.2-118.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:20 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

213 tok/s

Prefill

15,360 tok/s

Total

7,326 tok/s

TTFT

2,133 ms

Peak VRAM

5.1GB

Power draw

370.9W

Hardware cost

—

Prompt tokens

32763

Output tokens

512

Prefill tokens

32763

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32763 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32763. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 212.4-212.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:20 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

226 tok/s

Prefill

17,163 tok/s

Total

5,241 tok/s

TTFT

954 ms

Peak VRAM

5.1GB

Power draw

362.7W

Hardware cost

—

Prompt tokens

16381

Output tokens

512

Prefill tokens

16381

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16381 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16381. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 225.4-225.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:20 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

232 tok/s

Prefill

16,796 tok/s

Total

3,225 tok/s

TTFT

488 ms

Peak VRAM

5.1GB

Power draw

352.9W

Hardware cost

—

Prompt tokens

8191

Output tokens

512

Prefill tokens

8191

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8191 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8191. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 231.6-231.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:20 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

60.4 tok/s

Prefill

2,588 tok/s

Total

2,392 tok/s

TTFT

100,893 ms

Peak VRAM

4.4GB

Power draw

156W

Hardware cost

—

Prompt tokens

261095

Output tokens

512

Prefill tokens

261095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 261095 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 60.4-60.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

236 tok/s

Prefill

16,798 tok/s

Total

1,910 tok/s

TTFT

244 ms

Peak VRAM

5.1GB

Power draw

347.2W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 235.4-236.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

113 tok/s

Prefill

2,998 tok/s

Total

2,855 tok/s

TTFT

87,079 ms

Peak VRAM

4.6GB

Power draw

109.9W

Hardware cost

—

Prompt tokens

261095

Output tokens

512

Prefill tokens

261095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 261095 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 111.4-112.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

128 tok/s

Prefill

6,401 tok/s

Total

3,654 tok/s

TTFT

5,119 ms

Peak VRAM

4.4GB

Power draw

146.1W

Hardware cost

—

Prompt tokens

32763

Output tokens

512

Prefill tokens

32763

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32763 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32763. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 128.1-128.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

164 tok/s

Prefill

7,531 tok/s

Total

4,455 tok/s

TTFT

4,351 ms

Peak VRAM

4.6GB

Power draw

117.8W

Hardware cost

—

Prompt tokens

32763

Output tokens

512

Prefill tokens

32763

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32763 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32763. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 163.7-164.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

141 tok/s

Prefill

7,290 tok/s

Total

2,869 tok/s

TTFT

2,247 ms

Peak VRAM

4.4GB

Power draw

138.9W

Hardware cost

—

Prompt tokens

16381

Output tokens

512

Prefill tokens

16381

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3060 12GB

GPU slots

—

GPU count

1

VRAM

12GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Xeon E5-2680 v4 (14C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3060 12GB, stock 170 W limit, PCIe Gen3 x16 | Intel Xeon E5-2680 v4 14C | 32 GB DDR4-2400 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16381 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16381. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 140.6-140.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

239 tok/s

Prefill

14,675 tok/s

Total

1,120 tok/s

TTFT

140 ms

Peak VRAM

5.1GB

Power draw

344.6W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Q6_K

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-Q6_K_SLQ.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-Q6_K_SLQ.gguf (Q6_K). The Kaitchup GSQ-Q6_K_SLQ: SLQ + GSQ mixed precision (Q6_K 61%, Q8_0 36%, BF16 3%; SLQ and GSQ by IST-DASLab). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 237.1-239.0 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 4:03 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

169 tok/s

Prefill

8,249 tok/s

Total

3,373 tok/s

TTFT

1,986 ms

Peak VRAM

4.6GB

Power draw

107.1W

Hardware cost

—

Prompt tokens

16381

Output tokens

512

Prefill tokens

16381

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

IQ4_XS

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: LFM3-1.7B-GSQ-1GB-IQ4_XS_RCO.gguf (IQ4_XS). The Kaitchup GSQ-1GB-IQ4_XS_RCO: RCO + GSQ 1 GB mixed precision (Q4_K 28%, Q5_K 27%, IQ4_XS 27%, IQ3_S 8%, Q4_0 7%, Q6_K 2%). Part of a same-size comparison of LFM3-1.7B GGUFs on this rig (The Kaitchup SLQ and GSQ builds, AtomicChat AD-Q6_K, LiquidAI Q6_K). llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16381 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16381. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 169.4-170.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 4, 2026, 3:57 AM

Last edited

—

Model

kaitchup/LFM3-1.7B-GSQ-GGUF

Display name

LFM3-1.7B-GSQ-GGUF

Base model

LFM3-1.7B

Revision

73e4932a6a8a92fac1f59fc5f60679e70fc922ca

Family

Llama

Parameters

1.7B

Active params

—

MoE

no

Output

147 tok/s

Prefill

7,389 tok/s

Total

1,894 tok/s

TTFT

1,109 ms

Peak VRAM

4.4GB

Power draw

133.2W