DeepSeek-V4.1-Flash-1017 on CMP 170HX 64GB ×4

# x299 recipe as deployed (not an untuned baseline). Spec decode: none. Warm room, SYSTIN 29-30 C; instantaneous power peaks up to 228 W (GA100 reports instantaneous draw); enforced limit 180 W
# 4x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 per card | i7-9800X | 128 GB DDR4 | Ubuntu 26.04.1, driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image
# Checkpoint: hybrid built with Morrowmake's dsv41-flash-cmp170hx recipe: routed experts EXL3 2.9 bpw from LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw, all other weights DeepSeek's own FP8 from deepseek-ai/DeepSeek-V4.1-Flash-1017. Thanks to Morrowmake and LibertAIDAI.
vllm serve /models/DeepSeek-V4.1-Flash-1017-hybrid-exl3-fp8 --host 0.0.0.0 --served-model-name m --max-model-len 262144 --gpu-memory-utilization 0.95 --max-num-seqs 8 --max-num-batched-tokens 2048 --port 8000 --hf-overrides '{"engram_table_dir":"/models/DeepSeek-V4.1-Flash-1017-engram"}' --load-format exl3 --enable-expert-parallel --mm-encoder-tp-mode data --kv-cache-dtype bfloat16 --reasoning-parser deepseek_v41 --enable-auto-tool-choice --tool-call-parser deepseek_v41 --compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[2,3,4,5,6,8,9,10,12,14,15,16,18,20,21,24,25,28,30,32,35,36,40,42,48]}' --speculative-config '{"method":"dspark","num_speculative_tokens":7,"adaptive_k":{"by_load":[7,5,5,5,5,5,5,5,1,2,3,4,5],"draft_by_load":true},"enable_adaptive_verification":false}' --prefill-chunk-with-decodes 512 --exclude-tools-when-tool-choice-none --tensor-parallel-size 4 --pipeline-parallel-size 1 --speculative-config None

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.

TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.