入门指南排行榜Decode calculator模型OrganizationsReportsMatrix runs硬件基准评测市场租用ProAPI文档
语言
Actual Computer — Every computer, one endpoint
Sandberg

Sandberg

Verified

@Sandberg · Member since 2026

← Leaderboard
Verified 4w ago Mainly llama.cpp 200 approved runs

Submissions

3,189

200 approved public runs

Models

11

unique models

Avg output

82.9tok/s

Best 275 tok/s

Avg prefill

3,227tok/s

Best 18,370 tok/s

Avg total

943tok/s

Best 7,929 tok/s

Avg TTFT

38,338ms

Best 31.2 ms

Hardware

4

distinct setups

Matrix runs

351

speed test matrices

Matrix runs

Matrix runs 351

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
  • TwIL-8BCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · BF165 contexts × 8 concurrency levels
    Peak decode
    1,393.6 tok/s
    Peak prefill
    6,607 tok/s
  • TwIL-8BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    311.1 tok/s
    Peak prefill
    3,234 tok/s
  • Apex-35B-V1.0CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · BF166 contexts × 8 concurrency levels
    Peak decode
    2,901.5 tok/s
    Peak prefill
    11.7K tok/s
  • Carbon-XRTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    200.7 tok/s
    Peak prefill
    3,110 tok/s
  • Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    807.4 tok/s
    Peak prefill
    1,910 tok/s
  • Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    791.5 tok/s
    Peak prefill
    1,944 tok/s
  • Qwen3.5-122B-A10B-AWQ-4bitCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,613.3 tok/s
    Peak prefill
    5,659 tok/s
  • Qwen3.5-122B-A10B-NVFP4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,066.4 tok/s
    Peak prefill
    4,420 tok/s
  • Muse-Glimmer-30B-NVFP4-W4A4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP4 W4A45 contexts × 8 concurrency levels
    Peak decode
    1,320.9 tok/s
    Peak prefill
    1,954 tok/s
  • Thomson-1.0-Small-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,224.7 tok/s
    Peak prefill
    7,178 tok/s
See all 351 matrix runs
Personal bests

Personal bests11

Fastest approved result for each model.

1.7B22 runs
Output275 tok/s
Prefill6,456 tok/s
TTFT42.9 ms
Depth0k
RTX 3090 24GB

llama.cpp · IQ4_XS · 5d ago

65B8 runs
Output176 tok/s
Prefill667 tok/s
TTFT310 ms
Depth0k
CMP 170HX 64GB

vllm · GPTQ W4A16 · 2d ago

27B2 runs
Output168 tok/s
Prefill968 tok/s
TTFT306 ms
Depth0k
RTX 3090 24GB

vllm · EXL3 5.0 bpw (SC_5.00bpw_H6) · today

40B MoE32 runs
Output143 tok/s
Prefill1,183 tok/s
TTFT167 ms
Depth0k
RTX 3090 24GB

llama.cpp · Mix-12G · 4d ago

27B2 runs
Output141 tok/s
Prefill838 tok/s
TTFT276 ms
Depth0k
RTX 3090 24GB

llama.cpp · GGUF UD-Q4_K_M · today

3B MoE8 runs
Output131 tok/s
Prefill3,387 tok/s
TTFT68.8 ms
Depth0k
CMP 170HX 64GB ×2

vllm · BF16 · 5d ago

8B78 runs
Output122 tok/s
Prefill2,729 tok/s
TTFT84.6 ms
Depth0k
RTX 3090 24GB

llama.cpp · Q4_K_M · 4d ago

4B6 runs
Output118 tok/s
Prefill7,064 tok/s
TTFT290 ms
Depth2k
RTX 3090 24GB

llama.cpp · Q8_0 · 5d ago

33B2 runs
Output81.5 tok/s
Prefill2,735 tok/s
TTFT117 ms
Depth0k
RTX 3090 24GB

vllm · EXL3 4.00bpw H6 · today

65B32 runs
Output76.8 tok/s
Prefill174 tok/s
TTFT1,189 ms
Depth0k
CMP 170HX 64GB

llama.cpp · Q4_K_M · 3d ago

763B MoE8 runs
Output40.2 tok/s
Prefill171 tok/s
TTFT1,218 ms
Depth0k
CMP 170HX 64GB ×2

llama.cpp · IQ2_XXS · 5d ago

Hardware breakdown

Hardware breakdown4

Distinct rigs used across approved submissions.

RTX 3090 24GB

75 runs · best 275 tok/s

CMP 170HX 64GB

75 runs · best 176 tok/s

CMP 170HX 64GB ×2

32 runs · best 131 tok/s

RTX 3060 12GB

18 runs · best 147 tok/s

Engines used

Engines used2

Runtime mix and top quantizations.

llama.cpp164GGUF UD-Q4_K_M · Q8_0 · Q4_K_M
vllm36EXL3 5.0 bpw (SC_5.00bpw_H6) · EXL3 4.00bpw H6 · GPTQ W4A16
All runs

All runs200

Full approved speed test history · page 6 of 8.

Flint-abliterated-GGUF

40B MoE · Llama

88.8

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

21,987 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

88.8 tok/s

Prefill

1,490 tok/s

Total

1,199 tok/s

TTFT

21,987 ms

Peak VRAM

18.9GB

Power draw

145.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 88.7-88.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

91.6

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

10,768 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

91.6 tok/s

Prefill

1,521 tok/s

Total

1,033 tok/s

TTFT

10,768 ms

Peak VRAM

18.9GB

Power draw

134.7W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 91.2-91.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

93.5

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

5,487 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

93.5 tok/s

Prefill

1,493 tok/s

Total

794 tok/s

TTFT

5,487 ms

Peak VRAM

18.9GB

Power draw

125W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 93.4-93.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

94.5

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

2,959 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

94.5 tok/s

Prefill

1,384 tok/s

Total

550 tok/s

TTFT

2,959 ms

Peak VRAM

18.9GB

Power draw

109.3W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 94.5-94.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

95.3

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

1,697 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

95.3 tok/s

Prefill

1,206 tok/s

Total

362 tok/s

TTFT

1,697 ms

Peak VRAM

18.9GB

Power draw

102.2W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 95.2-95.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

63.1

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

258,411 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

63.1 tok/s

Prefill

1,010 tok/s

Total

982 tok/s

TTFT

258,411 ms

Peak VRAM

30GB

Power draw

160.4W

Hardware cost

—

Prompt tokens

261091

Output tokens

512

Prefill tokens

261091

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 261091 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261091. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 61.0-63.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

92.7

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

18,912 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

92.7 tok/s

Prefill

1,732 tok/s

Total

1,362 tok/s

TTFT

18,912 ms

Peak VRAM

30GB

Power draw

171.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 92.6-93.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

95.9

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

9,133 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

95.9 tok/s

Prefill

1,794 tok/s

Total

1,167 tok/s

TTFT

9,133 ms

Peak VRAM

30GB

Power draw

154.1W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 95.8-95.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

97.2

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

4,669 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

97.2 tok/s

Prefill

1,754 tok/s

Total

876 tok/s

TTFT

4,669 ms

Peak VRAM

30GB

Power draw

152.4W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 97.0-97.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

97.9

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

2,433 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

97.9 tok/s

Prefill

1,683 tok/s

Total

601 tok/s

TTFT

2,433 ms

Peak VRAM

30GB

Power draw

142.8W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 97.6-97.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

98.7

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

1,452 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

98.7 tok/s

Prefill

1,409 tok/s

Total

385 tok/s

TTFT

1,452 ms

Peak VRAM

30GB

Power draw

132.6W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 98.6-98.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:07 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

96.2

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

359 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

96.2 tok/s

Prefill

551 tok/s

Total

125 tok/s

TTFT

359 ms

Peak VRAM

18.9GB

Power draw

98.3W

Hardware cost

—

Prompt tokens

198

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 94.3-96.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:59 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

96.1

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-12G

TTFT

397 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

96.1 tok/s

Prefill

652 tok/s

Total

135 tok/s

TTFT

397 ms

Peak VRAM

18.9GB

Power draw

98.3W

Hardware cost

—

Prompt tokens

259

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 95.9-96.2 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:59 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

100

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

313 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

100 tok/s

Prefill

632 tok/s

Total

131 tok/s

TTFT

313 ms

Peak VRAM

30GB

Power draw

129.4W

Hardware cost

—

Prompt tokens

198

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 100.0-100.1 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:50 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

100

tok/s

Hardware

CMP 170HX 64GB

Engine

llama.cpp · Mix-24G

TTFT

346 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

100 tok/s

Prefill

748 tok/s

Total

141 tok/s

TTFT

346 ms

Peak VRAM

30GB

Power draw

129.4W

Hardware cost

—

Prompt tokens

259

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 99.9-100.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:50 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

61.9

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

173,888 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

61.9 tok/s

Prefill

1,502 tok/s

Total

1,436 tok/s

TTFT

173,888 ms

Peak VRAM

18.9GB

Power draw

406.6W

Hardware cost

—

Prompt tokens

261091

Output tokens

512

Prefill tokens

261091

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 261091 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261091. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 61.8-61.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

122

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

11,797 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

122 tok/s

Prefill

2,777 tok/s

Total

2,080 tok/s

TTFT

11,797 ms

Peak VRAM

18.9GB

Power draw

386.8W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 121.8-121.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

131

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

5,528 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

131 tok/s

Prefill

2,963 tok/s

Total

1,791 tok/s

TTFT

5,528 ms

Peak VRAM

18.9GB

Power draw

374.3W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 131.1-131.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

136

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

2,770 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

136 tok/s

Prefill

2,956 tok/s

Total

1,333 tok/s

TTFT

2,770 ms

Peak VRAM

18.9GB

Power draw

355.1W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 136.2-136.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

139

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

1,464 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

139 tok/s

Prefill

2,798 tok/s

Total

896 tok/s

TTFT

1,464 ms

Peak VRAM

18.9GB

Power draw

343.8W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 139.1-139.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

141

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

828 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

141 tok/s

Prefill

2,471 tok/s

Total

572 tok/s

TTFT

828 ms

Peak VRAM

18.9GB

Power draw

330.2W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 140.4-140.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

143

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

167 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

143 tok/s

Prefill

1,183 tok/s

Total

189 tok/s

TTFT

167 ms

Peak VRAM

18.9GB

Power draw

314.3W

Hardware cost

—

Prompt tokens

197

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 142.8-143.0 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 4:07 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

143

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-12G

TTFT

184 ms

Context

262k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

143 tok/s

Prefill

1,404 tok/s

Total

204 tok/s

TTFT

184 ms

Peak VRAM

18.9GB

Power draw

314.3W

Hardware cost

—

Prompt tokens

258

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 142.8-142.9 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 4:07 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

72.7

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-24G

TTFT

24,462 ms

Context

66k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

72.7 tok/s

Prefill

2,637 tok/s

Total

2,064 tok/s

TTFT

24,462 ms

Peak VRAM

23.3GB

Power draw

414W

Hardware cost

—

Prompt tokens

64502

Output tokens

512

Prefill tokens

64502

Context length

65536

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 65536 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, 65536 context (native 262144 does not fit with Q4_0 KV), prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 64502 tokens of context (63k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 64502. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 72.7-72.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:06 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

93.6

tok/s

Hardware

RTX 3090 24GB

Engine

llama.cpp · Mix-24G

TTFT

10,962 ms

Context

66k · 4d ago

Show all run details

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

93.6 tok/s

Prefill

2,989 tok/s

Total

2,025 tok/s

TTFT

10,962 ms

Peak VRAM

23.3GB

Power draw

413.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

65536

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 65536 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, 65536 context (native 262144 does not fit with Q4_0 KV), prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 93.5-93.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:06 AM

Last edited

—

ModelHardwareEnginetok/s outprefilltok/s totalTTFTDepthShare
Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G88.81,4901,19921,987 ms33k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

88.8 tok/s

Prefill

1,490 tok/s

Total

1,199 tok/s

TTFT

21,987 ms

Peak VRAM

18.9GB

Power draw

145.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 88.7-88.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G91.61,5211,03310,768 ms16k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

91.6 tok/s

Prefill

1,521 tok/s

Total

1,033 tok/s

TTFT

10,768 ms

Peak VRAM

18.9GB

Power draw

134.7W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 91.2-91.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G93.51,4937945,487 ms8k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

93.5 tok/s

Prefill

1,493 tok/s

Total

794 tok/s

TTFT

5,487 ms

Peak VRAM

18.9GB

Power draw

125W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 93.4-93.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G94.51,3845502,959 ms4k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

94.5 tok/s

Prefill

1,384 tok/s

Total

550 tok/s

TTFT

2,959 ms

Peak VRAM

18.9GB

Power draw

109.3W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 94.5-94.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G95.31,2063621,697 ms2k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

95.3 tok/s

Prefill

1,206 tok/s

Total

362 tok/s

TTFT

1,697 ms

Peak VRAM

18.9GB

Power draw

102.2W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 95.2-95.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:17 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G63.11,010982258,411 ms261k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

63.1 tok/s

Prefill

1,010 tok/s

Total

982 tok/s

TTFT

258,411 ms

Peak VRAM

30GB

Power draw

160.4W

Hardware cost

—

Prompt tokens

261091

Output tokens

512

Prefill tokens

261091

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 261091 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261091. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 61.0-63.6 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G92.71,7321,36218,912 ms33k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

92.7 tok/s

Prefill

1,732 tok/s

Total

1,362 tok/s

TTFT

18,912 ms

Peak VRAM

30GB

Power draw

171.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 92.6-93.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G95.91,7941,1679,133 ms16k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

95.9 tok/s

Prefill

1,794 tok/s

Total

1,167 tok/s

TTFT

9,133 ms

Peak VRAM

30GB

Power draw

154.1W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 95.8-95.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G97.21,7548764,669 ms8k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

97.2 tok/s

Prefill

1,754 tok/s

Total

876 tok/s

TTFT

4,669 ms

Peak VRAM

30GB

Power draw

152.4W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 97.0-97.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G97.91,6836012,433 ms4k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

97.9 tok/s

Prefill

1,683 tok/s

Total

601 tok/s

TTFT

2,433 ms

Peak VRAM

30GB

Power draw

142.8W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 97.6-97.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:08 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G98.71,4093851,452 ms2k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

98.7 tok/s

Prefill

1,409 tok/s

Total

385 tok/s

TTFT

1,452 ms

Peak VRAM

30GB

Power draw

132.6W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 98.6-98.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 12:07 PM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G96.2551125359 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

96.2 tok/s

Prefill

551 tok/s

Total

125 tok/s

TTFT

359 ms

Peak VRAM

18.9GB

Power draw

98.3W

Hardware cost

—

Prompt tokens

198

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 94.3-96.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:59 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-12G96.1652135397 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

96.1 tok/s

Prefill

652 tok/s

Total

135 tok/s

TTFT

397 ms

Peak VRAM

18.9GB

Power draw

98.3W

Hardware cost

—

Prompt tokens

259

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 95.9-96.2 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:59 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G100632131313 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

100 tok/s

Prefill

632 tok/s

Total

131 tok/s

TTFT

313 ms

Peak VRAM

30GB

Power draw

129.4W

Hardware cost

—

Prompt tokens

198

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 100.0-100.1 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:50 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

CMP 170HX 64GB
llama.cppMix-24G100748141346 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

100 tok/s

Prefill

748 tok/s

Total

141 tok/s

TTFT

346 ms

Peak VRAM

30GB

Power draw

129.4W

Hardware cost

—

Prompt tokens

259

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

CMP 170HX 64GB

GPU slots

—

GPU count

1

VRAM

64GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

2x Intel Xeon E5-2630 v3

RAM

64GB

OS

Ubuntu 26.04.1

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 99.9-100.4 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 11:50 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G61.91,5021,436173,888 ms261k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

61.9 tok/s

Prefill

1,502 tok/s

Total

1,436 tok/s

TTFT

173,888 ms

Peak VRAM

18.9GB

Power draw

406.6W

Hardware cost

—

Prompt tokens

261091

Output tokens

512

Prefill tokens

261091

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 261091 tokens of context (255k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 261091. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 61.8-61.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1222,7772,08011,797 ms33k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

122 tok/s

Prefill

2,777 tok/s

Total

2,080 tok/s

TTFT

11,797 ms

Peak VRAM

18.9GB

Power draw

386.8W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 121.8-121.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1312,9631,7915,528 ms16k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

131 tok/s

Prefill

2,963 tok/s

Total

1,791 tok/s

TTFT

5,528 ms

Peak VRAM

18.9GB

Power draw

374.3W

Hardware cost

—

Prompt tokens

16380

Output tokens

512

Prefill tokens

16380

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 131.1-131.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1362,9561,3332,770 ms8k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

136 tok/s

Prefill

2,956 tok/s

Total

1,333 tok/s

TTFT

2,770 ms

Peak VRAM

18.9GB

Power draw

355.1W

Hardware cost

—

Prompt tokens

8189

Output tokens

512

Prefill tokens

8189

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 136.2-136.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1392,7988961,464 ms4k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

139 tok/s

Prefill

2,798 tok/s

Total

896 tok/s

TTFT

1,464 ms

Peak VRAM

18.9GB

Power draw

343.8W

Hardware cost

—

Prompt tokens

4095

Output tokens

512

Prefill tokens

4095

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 139.1-139.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1412,471572828 ms2k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

141 tok/s

Prefill

2,471 tok/s

Total

572 tok/s

TTFT

828 ms

Peak VRAM

18.9GB

Power draw

330.2W

Hardware cost

—

Prompt tokens

2047

Output tokens

512

Prefill tokens

2047

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 140.4-140.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:19 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1431,183189167 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

143 tok/s

Prefill

1,183 tok/s

Total

189 tok/s

TTFT

167 ms

Peak VRAM

18.9GB

Power draw

314.3W

Hardware cost

—

Prompt tokens

197

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 142.8-143.0 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 4:07 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-12G1431,404204184 ms0k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

143 tok/s

Prefill

1,404 tok/s

Total

204 tok/s

TTFT

184 ms

Peak VRAM

18.9GB

Power draw

314.3W

Hardware cost

—

Prompt tokens

258

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-12G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

f16

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-12G.gguf -ngl 999 -c 262144 -np 1 -fa on -fit off

Extra flags

--fit off

Notes

GGUF file: Flint-abliterated-Mix-12G.gguf (Mix-12G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, F16 KV, 1 slot, native 262144 context, prompt cache off per request. Spec decode: none. KV cache: f16. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking off. Median of 5 runs (range 142.8-142.9 tok/s). tokSOut = (completion - 1) / time after first token.

Reactions

—

Submitted

Nov 5, 2026, 4:07 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-24G72.72,6372,06424,462 ms65k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

72.7 tok/s

Prefill

2,637 tok/s

Total

2,064 tok/s

TTFT

24,462 ms

Peak VRAM

23.3GB

Power draw

414W

Hardware cost

—

Prompt tokens

64502

Output tokens

512

Prefill tokens

64502

Context length

65536

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 65536 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, 65536 context (native 262144 does not fit with Q4_0 KV), prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 64502 tokens of context (63k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 64502. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 72.7-72.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:06 AM

Last edited

—

Flint-abliterated-GGUF

40B MoE · Llama

RTX 3090 24GB
llama.cppMix-24G93.62,9892,02510,962 ms33k
Show all details for Flint-abliterated-GGUF

Model

orcarouter/Flint-abliterated-GGUF

Display name

Flint-abliterated-GGUF

Base model

flint

Revision

4d618d1cb1d7782d4b08762fb664487bf5e4d3c4

Family

Llama

Parameters

40B

Active params

—

MoE

yes

Output

93.6 tok/s

Prefill

2,989 tok/s

Total

2,025 tok/s

TTFT

10,962 ms

Peak VRAM

23.3GB

Power draw

413.5W

Hardware cost

—

Prompt tokens

32762

Output tokens

512

Prefill tokens

32762

Context length

65536

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 24GB

GPU slots

—

GPU count

1

VRAM

24GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

Intel Core i7-9700K (8C)

RAM

32GB

OS

Ubuntu 24.04.5

Power

—

Engine

llama.cpp

Engine version

b12030

Quantization

Mix-24G

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

q4_0

KV cache size

—

Prefix caching

no

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

1

Scheduler delay

—

Num parallel

1

Concurrency

1

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

0

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Flint-abliterated-Mix-24G.gguf -ngl 999 -c 65536 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off

Extra flags

--ctk q4_0 --ctv q4_0 --fit off

Notes

GGUF file: Flint-abliterated-Mix-24G.gguf (Mix-24G). Flint-abliterated (abliterated RecursiveSI/flint, 40B MoE) GGUF by OrcaRouter, mixed-precision imatrix build. llama.cpp llama-server b12030 (official ghcr.io/ggml-org/llama.cpp server-cuda12 image), full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, 65536 context (native 262144 does not fit with Q4_0 KV), prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x RTX 3090 24GB (Zagitta/gpu-unlock), 420 W limit, PCIe Gen3 x16 | Intel Core i7-9700K 8C | 32 GB DDR4 | Ubuntu 24.04.5, NVIDIA driver 580.178.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking off. Median of 3 runs (range 93.5-93.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.

Reactions

—

Submitted

Nov 5, 2026, 4:06 AM

Last edited

—