GLM-5.3-569B-W4A16 on CMP 170HX 64GB ×8

# x299 recipe as deployed (not an untuned baseline). Spec decode: MTP, 3 tokens lookahead. Warm room, SYSTIN 29-31 C; instantaneous power peaks up to 322 W (GA100 reports instantaneous draw); enforced limit 180 W
# 8x NVIDIA CMP 170HX 64GB (cmpunlocker), 180 W limit, PCIe Gen2 x4 per card | i7-9800X | 128 GB DDR4 | Ubuntu 26.04.1, driver 610.57.04 | vLLM: Morrowmake vllm-cmp170hx image
# Checkpoint: W4H local derivative of 0xSero/GLM-5.3-569B-W4A16 (0xSero's W8 variant with the LM head re-quantized to int4 locally; REAP-pruned to 192 experts by 0xSero). Thanks to 0xSero. DSA attention on sm_80 via a local patch.
vllm serve /models/GLM-5.3-569B-W4A16 --served-model-name m --trust-remote-code --tensor-parallel-size 1 --pipeline-parallel-size 8 --max-model-len 1048576 --gpu-memory-utilization 0.95 --max-num-seqs 32 --reasoning-parser glm47 --tool-call-parser glm47 --enable-auto-tool-choice --host 0.0.0.0 --port 8000 --speculative-config '{"method":"mtp","num_speculative_tokens":3,"adaptive_k":{"by_load":[3,2],"draft_by_load":true}}' --kv-transfer-config '{"kv_connector":"OffloadingConnector","kv_role":"kv_both","kv_connector_extra_config":{"spec_name":"TieringOffloadingSpec","cpu_bytes_to_use":42949672960,"worker_kv_bytes_per_block":853248,"secondary_tiers":[{"type":"fs","root_dir":"/kvcache/glm53-569b-pp8/GLM-5.3-569B-W4A16-W4H"}]}}' --max-num-batched-tokens 1024 --long-prefill-token-threshold 0

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.

TTFT is measured after the benchmark warms the server's prompt cache, so it mostly excludes prefill.

Requests queued: the server could not run all requests at once, so some waited in a queue. Over capacity: not run because this many requests at this context would exceed the server's KV cache.