Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 351
Submitted by @Sandberg. Newest first. Page 15 of 15.- granite-4.2-3b-nvfp4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 3,799.2 tok/s
- Peak prefill
- 10.5K tok/s