Verified 4w ago Mainly llama.cpp 200 approved runs
Submissions
3,189
200 approved public runs
Models
11
unique models
Avg output
82.9tok/s
Best 275 tok/s
Avg prefill
3,227tok/s
Best 18,370 tok/s
Avg total
943tok/s
Best 7,929 tok/s
Avg TTFT
38,338ms
Best 31.2 ms
Hardware
4
distinct setups
Matrix runs
351
speed test matrices
Matrix runs351+
Matrix runs 351
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 15.4-15.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 14.2-14.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 131055 tokens of context (128k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 131055. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 8.7-8.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 5 runs (range 16.1-16.4 tok/s). tokSOut = (completion - 1) / time after first token. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch): on this code prompt the model went straight to code in all 5 runs, without reasoning first. Median of 5 runs (range 15.9-15.9 tok/s). tokSOut = (completion - 1) / time after first token. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 131055 tokens of context (128k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 131055. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 79 drafts, 1185 draft tokens, 439 accepted (37.0%). Median of 3 runs (range 79.1-85.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 77 drafts, 1155 draft tokens, 438 accepted (37.9%). Median of 3 runs (range 107.9-116.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 85 drafts, 1275 draft tokens, 435 accepted (34.1%). Median of 3 runs (range 101.8-114.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 98 drafts, 1470 draft tokens, 419 accepted (28.5%). Median of 3 runs (range 93.2-101.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 85 drafts, 1275 draft tokens, 426 accepted (33.4%). Median of 3 runs (range 110.6-112.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 79 drafts, 1185 draft tokens, 432 accepted (36.5%). Median of 3 runs (range 121.1-122.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 92 drafts, 1380 draft tokens, 425 accepted (30.8%). Median of 5 runs (range 96.4-115.8 tok/s). tokSOut = (completion - 1) / time after first token.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): at temperature 0 this code prompt is bimodal: answers that reason first run at about 85 tok/s, answers that go straight to code at about 180 tok/s. Here 4 of 5 runs went direct (our round-4 run: 2 of 5), so most of the change in this number vs round 4 comes from that split, not from kernel speed. Draft counters for the submitted run (/metrics deltas): 55 drafts, 825 draft tokens, 458 accepted (55.5%). Median of 5 runs (range 76.2-186.1 tok/s). tokSOut = (completion - 1) / time after first token.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 130031 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130031. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 80 drafts, 1178 draft tokens, 431 accepted (36.6%). Median of 3 runs (range 38.4-40.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1277 draft tokens, 425 accepted (33.3%). Median of 3 runs (range 45.1-53.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1269 draft tokens, 425 accepted (33.5%). Median of 3 runs (range 47.0-48.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1281 draft tokens, 425 accepted (33.2%). Median of 3 runs (range 48.6-49.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 77 drafts, 1107 draft tokens, 434 accepted (39.2%). Median of 3 runs (range 53.0-59.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 78 drafts, 1158 draft tokens, 433 accepted (37.4%). Median of 3 runs (range 52.1-60.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on: Ripple's chat template has no thinking switch, so the model decides for itself whether to reason before answering; the 512 measured tokens include any reasoning text. On this code prompt all 5 runs went straight to code without reasoning first. Draft counters for the submitted run (/metrics deltas): 57 drafts, 842 draft tokens, 454 accepted (53.9%). Median of 5 runs (range 76.4-83.7 tok/s). tokSOut = (completion - 1) / time after first token.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 82 drafts, 1207 draft tokens, 429 accepted (35.5%). Median of 5 runs (range 54.3-55.2 tok/s). tokSOut = (completion - 1) / time after first token.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 15.4-15.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 14.2-14.2 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 131055 tokens of context (128k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 131055. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 3 runs (range 8.7-8.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Median of 5 runs (range 16.1-16.4 tok/s). tokSOut = (completion - 1) / time after first token. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
llama-server -m Ripple-65B-Preview-Q4_K_M.gguf -ngl 999 -c 1048576 -ctk q4_0 -ctv q4_0 -np 1 -fa on -fit off
Extra flags
--ctk q4_0
--ctv q4_0
--fit off
Notes
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat; they may change once support is merged. llama.cpp llama-server built from ggml-org/llama.cpp PR #31692 (Ripple architecture support, not yet merged) at commit 57713fb on top of b12119, CUDA 12.8 server image built from the repo's .devops/cuda.Dockerfile for sm_80, full GPU offload, FlashAttention on, Q4_0 KV, 1 slot, native 1048576 context, prompt cache off per request. Spec decode: none. KV cache: q4_0. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch): on this code prompt the model went straight to code in all 5 runs, without reasoning first. Median of 5 runs (range 15.9-15.9 tok/s). tokSOut = (completion - 1) / time after first token. Reposted 2026-11-07 with corrected thinking wording; measured 2026-11-04.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 131055 tokens of context (128k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 131055. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 79 drafts, 1185 draft tokens, 439 accepted (37.0%). Median of 3 runs (range 79.1-85.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 77 drafts, 1155 draft tokens, 438 accepted (37.9%). Median of 3 runs (range 107.9-116.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 85 drafts, 1275 draft tokens, 435 accepted (34.1%). Median of 3 runs (range 101.8-114.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 98 drafts, 1470 draft tokens, 419 accepted (28.5%). Median of 3 runs (range 93.2-101.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 85 drafts, 1275 draft tokens, 426 accepted (33.4%). Median of 3 runs (range 110.6-112.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 79 drafts, 1185 draft tokens, 432 accepted (36.5%). Median of 3 runs (range 121.1-122.9 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): the model reasoned first in every run, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 92 drafts, 1380 draft tokens, 425 accepted (30.8%). Median of 5 runs (range 96.4-115.8 tok/s). tokSOut = (completion - 1) / time after first token.
Official vLLM entry, single-card tuned config (round 5, replaces our round-4 entry): builds on the official vLLM nightly 4fab5a07 (includes PR #65745, Ripple + DFlash2 support by the PR authors), ModelCloud/Ripple-65B-Preview-GPTQ-W4A16-v2 (ModelCloud's GPTQ W4A16 quant of ricursiveAI/Ripple-65B-Preview) and ricursiveAI's DFlash2 drafter (FP8 weight-only at load). Not reproducible from public pieces alone: adds private attention and scheduling patches (FlashInfer backend) that are not public. Thanks to ricursiveAI, ModelCloud and the vLLM PR #65745 authors. 1 card, --attention-backend FLASHINFER, max-num-seqs 8, prefix caching off, gpu-memory-utilization 0.96, 132096 context (native 1048576). Spec decode: DFlash2 drafter (ricursiveAI/Ripple-65B-Preview-DFlash2 @35609b3b, FP8 online), 15 tokens lookahead at batch 1-2 (11 at 3-4, 7 at 5-8). KV cache: auto (bf16). 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on (Ripple's chat template has no thinking switch; reasoning parsed by --reasoning-parser ripple): at temperature 0 this code prompt is bimodal: answers that reason first run at about 85 tok/s, answers that go straight to code at about 180 tok/s. Here 4 of 5 runs went direct (our round-4 run: 2 of 5), so most of the change in this number vs round 4 comes from that split, not from kernel speed. Draft counters for the submitted run (/metrics deltas): 55 drafts, 825 draft tokens, 458 accepted (55.5%). Median of 5 runs (range 76.2-186.1 tok/s). tokSOut = (completion - 1) / time after first token.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 130031 tokens of context (127k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 130031. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 80 drafts, 1178 draft tokens, 431 accepted (36.6%). Median of 3 runs (range 38.4-40.5 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 32762 tokens of context (32k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 32762. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1277 draft tokens, 425 accepted (33.3%). Median of 3 runs (range 45.1-53.7 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 16380 tokens of context (16k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 16380. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1269 draft tokens, 425 accepted (33.5%). Median of 3 runs (range 47.0-48.8 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 8189 tokens of context (8k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 8189. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 86 drafts, 1281 draft tokens, 425 accepted (33.2%). Median of 3 runs (range 48.6-49.1 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 4095 tokens of context (4k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 4095. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 77 drafts, 1107 draft tokens, 434 accepted (39.2%). Median of 3 runs (range 53.0-59.4 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Depth run: 2047 tokens of context (2k target). Method: system message = cache-bust nonce + filler (CPython 3.12 standard-library source, cut to length with the model's own tokenizer), then LocalMaxxing canonical prompt reasoning-v1 verbatim as the final user turn; the whole conversation is a cold prefill, so promptTokens = prefillTokens = 2047. Batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 78 drafts, 1158 draft tokens, 433 accepted (37.4%). Median of 3 runs (range 52.1-60.3 tok/s). tokSOut = (completion - 1) / time after first token; tokSPrefill = promptTokens / TTFT.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt code-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on: Ripple's chat template has no thinking switch, so the model decides for itself whether to reason before answering; the 512 measured tokens include any reasoning text. On this code prompt all 5 runs went straight to code without reasoning first. Draft counters for the submitted run (/metrics deltas): 57 drafts, 842 draft tokens, 454 accepted (53.9%). Median of 5 runs (range 76.4-83.7 tok/s). tokSOut = (completion - 1) / time after first token.
GGUF file: Ripple-65B-Preview-Q4_K_M.gguf (Q4_K_M). Ripple architecture is new: these numbers use the unmerged llama.cpp support from PR #31692 by notforu777, with GGUFs by AtomicChat and the DFlash2 drafter by ricursiveAI; they may change once support is merged. Single-card tuned config, posted next to our untuned baseline. llama.cpp llama-server built from llama.cpp PR #31692 + DFlash2 drafter with local fixes (NoPE layer, drafter hidden-state hook, sm_80 small-batch routing), CUDA 12.8, sm_80. Env: GGML_CUDA_MMQ_MAX_BATCH=64 GGML_CUDA_MMVQ_MAX_BATCH=2 GGML_CUDA_MMQ_SK_MULT=2. Full GPU offload (target and drafter), FlashAttention on, F16 KV, 1 slot, -ub 2048 -b 4096, 131072 context (native 1048576), prompt cache off per request. Spec decode: DFlash2 draft model (ricursiveAI/Ripple-65B-Preview-DFlash2, converted locally to a Q8_0 GGUF), 15 tokens lookahead (--spec-type draft-dflash --spec-draft-n-max 15). KV cache: f16. 1x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 | 2x Intel Xeon E5-2630 v3 | 64 GB DDR4 | Ubuntu 26.04.1, NVIDIA driver 610.57.04. Method: LocalMaxxing canonical prompt reasoning-v1, batch 1, streaming, temperature 0, 512 output tokens (ignore_eos), cache-bust nonce so every prefill is cold, thinking left on: Ripple's chat template has no thinking switch and the model always reasons first, so the 512 measured tokens include its reasoning text. Draft counters for the submitted run (/metrics deltas): 82 drafts, 1207 draft tokens, 429 accepted (35.5%). Median of 5 runs (range 54.3-55.2 tok/s). tokSOut = (completion - 1) / time after first token.
