A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 1474.6 tok/s is the aggregate decode rate across all 16 streams, about 92.2 tok/s per stream, and the 438 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 1483.8, 1465.5, 1474.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 2505.8 tok/s is the aggregate decode rate across all 128 streams, about 19.6 tok/s per stream, and the 2240 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 2510.9, 2503.6, 2505.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 496.2 tok/s is the aggregate decode rate across all 8 streams, about 62.0 tok/s per stream, and the 211 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 496, 496.3, 496.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 5.8 tok/s is the aggregate decode rate across all 4 streams, about 1.4 tok/s per stream, and the 7168 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47870 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.8, 8.4, 8.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1933.4 tok/s is the aggregate decode rate across all 64 streams, about 30.2 tok/s per stream, and the 1081 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1932.7, 1935.3, 1933.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 268.8 tok/s is the aggregate decode rate across all 4 streams, about 67.2 tok/s per stream, and the 135 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 269.1, 268.6, 268.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 1371.7 tok/s is the aggregate decode rate across all 32 streams, about 42.9 tok/s per stream, and the 552 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 1372.9, 1365.3, 1371.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 2565.9 tok/s is the aggregate decode rate across all 256 streams, about 10.0 tok/s per stream, and the 4026 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 2562.8, 2566.1, 2565.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 876.5 tok/s is the aggregate decode rate across all 16 streams, about 54.8 tok/s per stream, and the 382 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 876.5, 874.5, 878.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 2363.6 tok/s is the aggregate decode rate across all 128 streams, about 18.5 tok/s per stream, and the 1920 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 2362.1, 2363.6, 2364.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 541.1 tok/s is the aggregate decode rate across all 8 streams, about 67.6 tok/s per stream, and the 285 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (298 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 553.5, 541.1, 562.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1745.4 tok/s is the aggregate decode rate across all 64 streams, about 27.3 tok/s per stream, and the 1214 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1785.1, 1752.8, 1745.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 61.2 tok/s is the aggregate decode rate across all 4 streams, about 15.3 tok/s per stream, and the 1560 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 61.1, 61.2, 61.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 115.5 tok/s is the aggregate decode rate across all 8 streams, about 14.4 tok/s per stream, and the 3062 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 115.7, 115.5, 115.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 60.6 tok/s is the aggregate decode rate across all 4 streams, about 15.2 tok/s per stream, and the 1562 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 60.3, 60.6, 61.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 259.2 tok/s is the aggregate decode rate across all 32 streams, about 8.1 tok/s per stream, and the 9210 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 260, 260.1, 259.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 32.1 tok/s is the aggregate decode rate across all 2 streams, about 16.1 tok/s per stream, and the 647 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (302 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 32.4, 32.1, 32.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 187 tok/s is the aggregate decode rate across all 16 streams, about 11.7 tok/s per stream, and the 5085 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 189.8, 187, 187.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47870 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 9.3, 9.5, 9.7 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config vl31-70b-awq, vLLM 0.32.0. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 16.6, 16.4, 16.6 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 315.6 tok/s is the aggregate decode rate across all 8 streams, about 39.5 tok/s per stream, and the 1466 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 315.6, 317.3, 308 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 269.4 tok/s is the aggregate decode rate across all 4 streams, about 67.3 tok/s per stream, and the 737 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 269.4, 268.6, 268.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 190.7 tok/s is the aggregate decode rate across all 2 streams, about 95.3 tok/s per stream, and the 432 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 189.9, 191.3, 190.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 23 tok/s is the aggregate decode rate across all 4 streams, about 5.8 tok/s per stream, and the 3321 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 104 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47716 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 23, 27.8, 26.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 112.1, 111.5, 110.6 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 1474.6 tok/s is the aggregate decode rate across all 16 streams, about 92.2 tok/s per stream, and the 438 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 1483.8, 1465.5, 1474.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 2505.8 tok/s is the aggregate decode rate across all 128 streams, about 19.6 tok/s per stream, and the 2240 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 2510.9, 2503.6, 2505.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 496.2 tok/s is the aggregate decode rate across all 8 streams, about 62.0 tok/s per stream, and the 211 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 496, 496.3, 496.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 5.8 tok/s is the aggregate decode rate across all 4 streams, about 1.4 tok/s per stream, and the 7168 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47870 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.8, 8.4, 8.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1933.4 tok/s is the aggregate decode rate across all 64 streams, about 30.2 tok/s per stream, and the 1081 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1932.7, 1935.3, 1933.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 268.8 tok/s is the aggregate decode rate across all 4 streams, about 67.2 tok/s per stream, and the 135 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 269.1, 268.6, 268.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 1371.7 tok/s is the aggregate decode rate across all 32 streams, about 42.9 tok/s per stream, and the 552 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 1372.9, 1365.3, 1371.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 2565.9 tok/s is the aggregate decode rate across all 256 streams, about 10.0 tok/s per stream, and the 4026 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 2562.8, 2566.1, 2565.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 876.5 tok/s is the aggregate decode rate across all 16 streams, about 54.8 tok/s per stream, and the 382 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 876.5, 874.5, 878.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 2363.6 tok/s is the aggregate decode rate across all 128 streams, about 18.5 tok/s per stream, and the 1920 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 2362.1, 2363.6, 2364.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq4-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 541.1 tok/s is the aggregate decode rate across all 8 streams, about 67.6 tok/s per stream, and the 285 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (298 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 553.5, 541.1, 562.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1745.4 tok/s is the aggregate decode rate across all 64 streams, about 27.3 tok/s per stream, and the 1214 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1785.1, 1752.8, 1745.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 61.2 tok/s is the aggregate decode rate across all 4 streams, about 15.3 tok/s per stream, and the 1560 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 61.1, 61.2, 61.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 115.5 tok/s is the aggregate decode rate across all 8 streams, about 14.4 tok/s per stream, and the 3062 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 115.7, 115.5, 115.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 60.6 tok/s is the aggregate decode rate across all 4 streams, about 15.2 tok/s per stream, and the 1562 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 60.3, 60.6, 61.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 259.2 tok/s is the aggregate decode rate across all 32 streams, about 8.1 tok/s per stream, and the 9210 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 260, 260.1, 259.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 32.1 tok/s is the aggregate decode rate across all 2 streams, about 16.1 tok/s per stream, and the 647 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (302 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 32.4, 32.1, 32.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 187 tok/s is the aggregate decode rate across all 16 streams, about 11.7 tok/s per stream, and the 5085 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 189.8, 187, 187.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47870 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 9.3, 9.5, 9.7 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config vl31-70b-awq, vLLM 0.32.0. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 16.6, 16.4, 16.6 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 315.6 tok/s is the aggregate decode rate across all 8 streams, about 39.5 tok/s per stream, and the 1466 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 315.6, 317.3, 308 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 269.4 tok/s is the aggregate decode rate across all 4 streams, about 67.3 tok/s per stream, and the 737 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 269.4, 268.6, 268.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 190.7 tok/s is the aggregate decode rate across all 2 streams, about 95.3 tok/s per stream, and the 432 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 189.9, 191.3, 190.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 23 tok/s is the aggregate decode rate across all 4 streams, about 5.8 tok/s per stream, and the 3321 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 104 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47716 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 23, 27.8, 26.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 112.1, 111.5, 110.6 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
