neko-legends

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

46.2 tok/s

Prefill

777 tok/s

Total

—

TTFT

311 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

242

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

46.2 tok/s

Prefill

777 tok/s

Total

—

TTFT

311 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

242

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

72.0 tok/s

Prefill

391 tok/s

Total

—

TTFT

471 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

184

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

72.0 tok/s

Prefill

391 tok/s

Total

—

TTFT

471 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

184

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

46.2 tok/s

Prefill

777 tok/s

Total

—

TTFT

311 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

242

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—

Model

deepseek-ai/DeepSeek-V4.1-Flash

Display name

DeepSeek-V4.1-Flash

Base model

—

Revision

fb2764a5cf321eaa5070ca8f9e892818f477c16d

Family

Deepseek

Parameters

763B

Active params

—

MoE

no

Output

72.0 tok/s

Prefill

391 tok/s

Total

—

TTFT

471 ms

Peak VRAM

—

Power draw

96W (24 + 23 + 25 + 24W)

Hardware cost

—

Prompt tokens

184

Output tokens

600

Prefill tokens

0

Context length

430080

Batch size

1

Hardware class

UNIFIED

Hardware

GB10 Grace Blackwell 512GB

GPU slots

—

GPU count

1

VRAM

—

Chip vendor

NVIDIA

Chip family

GB10

Chip variant

GB10 Grace Blackwell

Unified memory

512GB

NPU TOPS

—

CPU

—

RAM

—

OS

Ubuntu 24.04 (DGX OS)

Power

—

Engine

vllm

Engine version

dsv41-feat e47aa780

Quantization

NVFP4

Backend

cuda

Tensor parallel

4

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

fp8

KV cache size

—

Prefix caching

yes

Attention backend

—

Flash attention

—

Chunked prefill

yes

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

0.8

Max running seqs

8

Scheduler delay

—

Num parallel

—

Concurrency

1

Spec decoding

yes

Spec method

dspark

Spec model

—

Spec draft model

—

Spec tokens

5

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

MAXLEN=430080 GMU=0.80 SEQS=8 SPEC_K=5 DRAFT_METHOD=greedy ENGRAM_THREADS=32 MAX_BATCHED=16384 bash launch-dsv41-vllm-tp4.sh # TP4 across 4 nodes, workers-first, head last; full launcher: github.com/neko-legends/spark-bench

Extra flags

--speculative-config {method:dspark,num_speculative_tokens:5,draft_sample_method:greedy} --compilation-config {cudagraph_mode:FULL_AND_PIECEWISE} --block-size 128 --max-num-batched-tokens 16384 --max-num-seqs 8 --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41 --distributed-executor-backend mp --nnodes 4

Notes

4x DGX Spark (GB10), TP4 over ConnectX-7 RoCE, one vLLM world on :8000. Native FP4 routed experts + FP8 dense + FP8 Engram served from local NVMe per node (engram-on-disk patch, tonyd2wild/Kai). DSpark speculative decoding k=5, greedy draft. Context 420k. Bench recipe: canonical prompt, temp 0, thinking off, batch 1, after warmup.

Reactions

—

Submitted

Sep 11, 2026, 7:43 AM

Last edited

—