시작하기리더보드Decode calculator모델OrganizationsReportsMatrix runs하드웨어벤치마크마켓플레이스렌탈ProAPI 문서
언어
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 11 of 15.
  • Qwen3.5-27B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit4 contexts × 8 concurrency levels
    Peak decode
    180.5 tok/s
    Peak prefill
    1,804 tok/s
  • Tess-4-27B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    187.2 tok/s
    Peak prefill
    1,483 tok/s
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A165 contexts × 8 concurrency levels
    Peak decode
    366.7 tok/s
    Peak prefill
    1,808 tok/s
  • Sage-A1.5-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,054.1 tok/s
    Peak prefill
    7,513 tok/s
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    985.1 tok/s
    Peak prefill
    4,078 tok/s
  • gemma-4-31B-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    557.2 tok/s
    Peak prefill
    1,519 tok/s
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1285 contexts × 8 concurrency levels
    Peak decode
    391.8 tok/s
    Peak prefill
    1,500 tok/s
  • granite-4.2-30b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    152.7 tok/s
    Peak prefill
    1,365 tok/s
  • Llama-3.3-70B-Instruct-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    165.0 tok/s
    Peak prefill
    702.0 tok/s
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    993.9 tok/s
    Peak prefill
    9,721 tok/s
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,719.6 tok/s
    Peak prefill
    10.8K tok/s
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A163 contexts × 8 concurrency levels
    Peak decode
    170.6 tok/s
    Peak prefill
    2,162 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    364.7 tok/s
    Peak prefill
    8,151 tok/s
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1284 contexts × 8 concurrency levels
    Peak decode
    120.9 tok/s
    Peak prefill
    1,749 tok/s
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,671.4 tok/s
    Peak prefill
    6,435 tok/s
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,137.8 tok/s
    Peak prefill
    5,892 tok/s
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,397.0 tok/s
    Peak prefill
    3,012 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,791.8 tok/s
    Peak prefill
    8,758 tok/s
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ5 contexts × 8 concurrency levels
    Peak decode
    777.1 tok/s
    Peak prefill
    2,138 tok/s
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,407.4 tok/s
    Peak prefill
    2,331 tok/s
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    814.6 tok/s
    Peak prefill
    2,365 tok/s
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,551.0 tok/s
    Peak prefill
    4,074 tok/s
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1283 contexts × 8 concurrency levels
    Peak decode
    167.0 tok/s
    Peak prefill
    1,845 tok/s
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,274.3 tok/s
    Peak prefill
    6,070 tok/s
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    1,766.5 tok/s
    Peak prefill
    9,483 tok/s