A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 82.5 tok/s is the aggregate decode rate across all 16 streams, about 5.2 tok/s per stream, and the 11107 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47374 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 82.5, 133.3, 69.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 46.5 tok/s is the aggregate decode rate across all 16 streams, about 2.9 tok/s per stream, and the 7375 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47540 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.4, 46.5, 66.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 169.8 tok/s is the aggregate decode rate across all 16 streams, about 10.6 tok/s per stream, and the 6561 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47460 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 181.5, 175.2, 169.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 63 tok/s is the aggregate decode rate across all 16 streams, about 3.9 tok/s per stream, and the 8097 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47320 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 63, 63.1, 70 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 484.6 tok/s is the aggregate decode rate across all 8 streams, about 60.6 tok/s per stream, and the 390 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 484.6, 528.7, 473.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1447.8 tok/s is the aggregate decode rate across all 64 streams, about 22.6 tok/s per stream, and the 2061 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1470.4, 1427.4, 1447.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 288.1 tok/s is the aggregate decode rate across all 4 streams, about 72.0 tok/s per stream, and the 243 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 288.1, 297.7, 302.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47476 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 1 slot, context 32768. MTP speculative decoding from mtp-Qwen3.8-Flash-Next-Q8_0.gguf, up to 2 draft tokens. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.9, 52.8, 55.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 9.5 tok/s is the aggregate decode rate across all 8 streams, about 1.2 tok/s per stream, and the 4499 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.2, 9.5, 8.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 1236.8 tok/s is the aggregate decode rate across all 32 streams, about 38.6 tok/s per stream, and the 1096 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 1240.5, 1228.2, 1236.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 1548.6 tok/s is the aggregate decode rate across all 256 streams, about 6.0 tok/s per stream, and the 18549 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 1548.6, 1561.6, 1547.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 158.1 tok/s is the aggregate decode rate across all 2 streams, about 79.0 tok/s per stream, and the 128 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 160.4, 158.1, 147.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 814.2 tok/s is the aggregate decode rate across all 16 streams, about 50.9 tok/s per stream, and the 726 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 814.2, 825.5, 803.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 1434.6 tok/s is the aggregate decode rate across all 128 streams, about 11.2 tok/s per stream, and the 3819 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 1434.6, 1429.7, 1436.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 292.6 tok/s is the aggregate decode rate across all 8 streams, about 36.6 tok/s per stream, and the 364 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 292.6, 293, 292.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1241.9 tok/s is the aggregate decode rate across all 64 streams, about 19.4 tok/s per stream, and the 1867 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1241.9, 1241, 1242.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 152 tok/s is the aggregate decode rate across all 4 streams, about 38.0 tok/s per stream, and the 231 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 151.9, 152, 152.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6.5 tok/s is the aggregate decode rate across all 4 streams, about 1.6 tok/s per stream, and the 5300 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.1, 6.5, 6.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 5.9 tok/s is the aggregate decode rate across all 2 streams, about 3.0 tok/s per stream, and the 3550 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 6.3, 11.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 845 tok/s is the aggregate decode rate across all 32 streams, about 26.4 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 844.8, 846.4, 845 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 1745.8 tok/s is the aggregate decode rate across all 256 streams, about 6.8 tok/s per stream, and the 6884 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 1747, 1744.3, 1745.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 77.9 tok/s is the aggregate decode rate across all 2 streams, about 39.0 tok/s per stream, and the 110 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 77.9, 77.9, 78 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 26.1 tok/s is the aggregate decode rate across all 8 streams, about 3.3 tok/s per stream, and the 4686 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 74 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-MXFP4_MOE-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 27.4, 27, 26.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 17.9, 17.5, 19.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 522.6 tok/s is the aggregate decode rate across all 16 streams, about 32.7 tok/s per stream, and the 668 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 523.4, 523.2, 522.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 82.5 tok/s is the aggregate decode rate across all 16 streams, about 5.2 tok/s per stream, and the 11107 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47374 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 82.5, 133.3, 69.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 46.5 tok/s is the aggregate decode rate across all 16 streams, about 2.9 tok/s per stream, and the 7375 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47540 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.4, 46.5, 66.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 169.8 tok/s is the aggregate decode rate across all 16 streams, about 10.6 tok/s per stream, and the 6561 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47460 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 181.5, 175.2, 169.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 63 tok/s is the aggregate decode rate across all 16 streams, about 3.9 tok/s per stream, and the 8097 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47320 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 63, 63.1, 70 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 484.6 tok/s is the aggregate decode rate across all 8 streams, about 60.6 tok/s per stream, and the 390 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 484.6, 528.7, 473.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1447.8 tok/s is the aggregate decode rate across all 64 streams, about 22.6 tok/s per stream, and the 2061 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1470.4, 1427.4, 1447.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 288.1 tok/s is the aggregate decode rate across all 4 streams, about 72.0 tok/s per stream, and the 243 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 288.1, 297.7, 302.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47476 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 1 slot, context 32768. MTP speculative decoding from mtp-Qwen3.8-Flash-Next-Q8_0.gguf, up to 2 draft tokens. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.9, 52.8, 55.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 9.5 tok/s is the aggregate decode rate across all 8 streams, about 1.2 tok/s per stream, and the 4499 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.2, 9.5, 8.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 1236.8 tok/s is the aggregate decode rate across all 32 streams, about 38.6 tok/s per stream, and the 1096 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 1240.5, 1228.2, 1236.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 1548.6 tok/s is the aggregate decode rate across all 256 streams, about 6.0 tok/s per stream, and the 18549 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 1548.6, 1561.6, 1547.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 158.1 tok/s is the aggregate decode rate across all 2 streams, about 79.0 tok/s per stream, and the 128 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 160.4, 158.1, 147.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 814.2 tok/s is the aggregate decode rate across all 16 streams, about 50.9 tok/s per stream, and the 726 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 814.2, 825.5, 803.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 128 requests sent at once (batch 128, lmx --concurrency 128); 1434.6 tok/s is the aggregate decode rate across all 128 streams, about 11.2 tok/s per stream, and the 3819 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 128: 1434.6, 1429.7, 1436.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16-mtp2, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 292.6 tok/s is the aggregate decode rate across all 8 streams, about 36.6 tok/s per stream, and the 364 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 292.6, 293, 292.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 64 requests sent at once (batch 64, lmx --concurrency 64); 1241.9 tok/s is the aggregate decode rate across all 64 streams, about 19.4 tok/s per stream, and the 1867 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 64: 1241.9, 1241, 1242.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 152 tok/s is the aggregate decode rate across all 4 streams, about 38.0 tok/s per stream, and the 231 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 151.9, 152, 152.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6.5 tok/s is the aggregate decode rate across all 4 streams, about 1.6 tok/s per stream, and the 5300 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.1, 6.5, 6.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 5.9 tok/s is the aggregate decode rate across all 2 streams, about 3.0 tok/s per stream, and the 3550 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 6.3, 11.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 845 tok/s is the aggregate decode rate across all 32 streams, about 26.4 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 844.8, 846.4, 845 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 256 requests sent at once (batch 256, lmx --concurrency 256); 1745.8 tok/s is the aggregate decode rate across all 256 streams, about 6.8 tok/s per stream, and the 6884 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 256: 1747, 1744.3, 1745.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (batch 2, lmx --concurrency 2); 77.9 tok/s is the aggregate decode rate across all 2 streams, about 39.0 tok/s per stream, and the 110 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 2: 77.9, 77.9, 78 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 26.1 tok/s is the aggregate decode rate across all 8 streams, about 3.3 tok/s per stream, and the 4686 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 74 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-MXFP4_MOE-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 27.4, 27, 26.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 17.9, 17.5, 19.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 522.6 tok/s is the aggregate decode rate across all 16 streams, about 32.7 tok/s per stream, and the 668 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 523.4, 523.2, 522.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq9-bf16, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
