入门指南排行榜Decode calculator模型OrganizationsReportsMatrix runs硬件基准评测市场租用ProAPI文档
语言
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 8 of 15.
  • Qwen-AgentWorld-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,849.5 tok/s
    Peak prefill
    7,749 tok/s
  • Qwen3-1.7B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    3,788.8 tok/s
    Peak prefill
    18K tok/s
  • Tess-4-9BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    378.8 tok/s
    Peak prefill
    7,169 tok/s
  • Qwen3-8B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,663.1 tok/s
    Peak prefill
    3,384 tok/s
  • Llama-3.2-3B-Instruct-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    2,507.0 tok/s
    Peak prefill
    13.6K tok/s
  • DeepSeek-R1-Distill-Qwen-32B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    244.1 tok/s
    Peak prefill
    1,856 tok/s
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    3,794.2 tok/s
    Peak prefill
    8,459 tok/s
  • Qwen3-0.6B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    1,792.5 tok/s
    Peak prefill
    37.9K tok/s
  • Fara1.5-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    4,940.8 tok/s
    Peak prefill
    10.3K tok/s
  • Seed-OSS-36B-Instruct-FP8-StaticCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 static5 contexts × 8 concurrency levels
    Peak decode
    224.8 tok/s
    Peak prefill
    1,545 tok/s
  • Qwen3-1.7B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    684.9 tok/s
    Peak prefill
    21.8K tok/s
  • granite-4.2-3b-fp8RTX 3090 24GB · vllm 0.32.0 · FP85 contexts × 8 concurrency levels
    Peak decode
    1,601.6 tok/s
    Peak prefill
    12.3K tok/s
  • Agents-A1-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    4,812.8 tok/s
    Peak prefill
    10.8K tok/s
  • Qwen3-0.6B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    3,926.4 tok/s
    Peak prefill
    27.5K tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,858.9 tok/s
    Peak prefill
    8,430 tok/s
  • granite-4.2-3b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    3,789.9 tok/s
    Peak prefill
    11.4K tok/s
  • Llama-3.2-3B-Instruct-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,514.0 tok/s
    Peak prefill
    14.6K tok/s
  • Ministral-3-3B-Instruct-2512-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,735.6 tok/s
    Peak prefill
    11.5K tok/s
  • NVIDIA-Nemotron-3-Nano-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,664.8 tok/s
    Peak prefill
    10.4K tok/s
  • NVIDIA-Nemotron-3-Nano-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,229.6 tok/s
    Peak prefill
    11.6K tok/s
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    929.7 tok/s
    Peak prefill
    7,033 tok/s
  • granite-4.2-8b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    1,636.8 tok/s
    Peak prefill
    6,024 tok/s
  • Qwen3.5-4B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    3,662.7 tok/s
    Peak prefill
    10.6K tok/s
  • NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    2,101.8 tok/s
    Peak prefill
    6,307 tok/s
  • Fara1.5-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,189.2 tok/s
    Peak prefill
    11.8K tok/s