MulaiPapan peringkatDecode calculatorModelOrganizationsReportsMatrix runsHardwareTolok UkurMarketplaceSewaProDokumen API
Bahasa
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 6 of 15.
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A165 contexts × 8 concurrency levels
    Peak decode
    394.4 tok/s
    Peak prefill
    1,537 tok/s
  • Llama-3.2-3B-Instruct-FP8-dynamicRTX 3060 12GB · vllm 0.32.0 · FP8 dynamic4 contexts × 8 concurrency levels
    Peak decode
    267.2 tok/s
    Peak prefill
    3,265 tok/s
  • Step3-VL-10BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    200.5 tok/s
    Peak prefill
    6,946 tok/s
  • granite-4.2-3b-fp8RTX 3060 12GB · vllm 0.32.0 · FP84 contexts × 8 concurrency levels
    Peak decode
    478.0 tok/s
    Peak prefill
    2,841 tok/s
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    259.9 tok/s
    Peak prefill
    6,640 tok/s
  • Qwen3-1.7B-FP8RTX 3060 12GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    564.3 tok/s
    Peak prefill
    5,516 tok/s
  • Qwen3-0.6B-FP8RTX 3060 12GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    665.4 tok/s
    Peak prefill
    9,704 tok/s
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    378.8 tok/s
    Peak prefill
    6,966 tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    678.0 tok/s
    Peak prefill
    2,654 tok/s
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    933.6 tok/s
    Peak prefill
    3,531 tok/s
  • gemma-4-E4B-itRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    563.3 tok/s
    Peak prefill
    10.2K tok/s
  • Ornith-1.5-9B-NVFP4RTX 3060 12GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    367.1 tok/s
    Peak prefill
    1,599 tok/s
  • Qwen3.5-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,039.0 tok/s
    Peak prefill
    11.1K tok/s
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3060 12GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    480.5 tok/s
    Peak prefill
    1,337 tok/s
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    974.8 tok/s
    Peak prefill
    9,139 tok/s
  • Qwen3.5-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,836.2 tok/s
    Peak prefill
    11.8K tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    812.4 tok/s
    Peak prefill
    2,765 tok/s
  • granite-4.2-3bRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,222.5 tok/s
    Peak prefill
    12.7K tok/s
  • DeepSeek-R1-Distill-Qwen-14B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    844.7 tok/s
    Peak prefill
    3,910 tok/s
  • MiniCPM5-1BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    4,618.6 tok/s
    Peak prefill
    36.8K tok/s
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3060 12GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    820.4 tok/s
    Peak prefill
    2,790 tok/s
  • granite-4.2-3b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    591.9 tok/s
    Peak prefill
    2,775 tok/s
  • NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP4 mixed6 contexts × 8 concurrency levels
    Peak decode
    1,078.6 tok/s
    Peak prefill
    3,902 tok/s
  • Qwen3-Next-80B-A3B-Thinking-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    2,121.7 tok/s
    Peak prefill
    7,388 tok/s
  • Ornith-1.5-35B-A3B-W4A16-SYMCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A166 contexts × 8 concurrency levels
    Peak decode
    3,131.6 tok/s
    Peak prefill
    7,431 tok/s