GLM-5.3-Flash-W4A16-MTP on CMP 170HX 64GB ×4

# x299 recipe as deployed (not an untuned baseline). Spec decode: DFLASH, 7 tokens lookahead. Warm room, SYSTIN 28-30 C; instantaneous power peaks up to 344 W (GA100 reports instantaneous draw); enforced limit 180 W
# 4x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 per card | i7-9800X | 128 GB DDR4 | Ubuntu 26.04.1, driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image
vllm serve /models/GLM-5.3-Flash-W4A16-MTP --served-model-name m --host 0.0.0.0 --pipeline-parallel-size 4 --tensor-parallel-size 1 --max-model-len 524288 --max-num-seqs 8 --max-num-batched-tokens 2312 --long-prefill-token-threshold 0 --gpu-memory-utilization 0.95 --trust-remote-code --reasoning-parser glm47 --enable-auto-tool-choice --tool-call-parser glm47 --port 8000 --speculative-config '{"method":"dflash","model":"/models/GLM-5.3-Flash-DFlash2","num_speculative_tokens":7,"adaptive_k":{"by_load":[7,5,3],"accept":{"costs":[1.0,1.0894,1.1375,1.2001,1.2239],"costs_multi":[1.0,1.0889,1.1194,1.2001,1.2239],"hysteresis":0.03},"log_interval":0}}' --prefill-chunk-with-decodes 384 --max-num-partial-prefills 2 --block-size 4608

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.

TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.