Bắt đầuBảng xếp hạngDecode calculatorMô hìnhOrganizationsReportsMatrix runsPhần cứngĐánh giá chuẩnChợCho thuêProTài liệu API
Ngôn ngữ
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 693

Submitted by @keennay. Newest first. Page 8 of 28.
  • granite-4.2-8bRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF165 contexts × 8 concurrency levels
    Peak decode
    866.9 tok/s
    Peak prefill
    17.7K tok/s
  • granite-4.2-3bRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev220+gfa7784d14 · BF165 contexts × 8 concurrency levels
    Peak decode
    3,350.4 tok/s
    Peak prefill
    37K tok/s
  • gemma-4-E4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    2,697.5 tok/s
    Peak prefill
    27.1K tok/s
  • gemma-4-E2B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    9,035.1 tok/s
    Peak prefill
    39.8K tok/s
  • gemma-4-26B-A4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,702.1 tok/s
    Peak prefill
    21.9K tok/s
  • gemma-4-12B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    719.3 tok/s
    Peak prefill
    11K tok/s
  • diffusiongemma-26B-A4B-itRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,469.4 tok/s
    Peak prefill
    10.7K tok/s
  • gemma-4-E4B-itRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF165 contexts × 8 concurrency levels
    Peak decode
    152.4 tok/s
    Peak prefill
    22.9K tok/s
  • gemma-4-E2B-itRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF165 contexts × 8 concurrency levels
    Peak decode
    7,851.5 tok/s
    Peak prefill
    33.5K tok/s
  • gemma-4-12B-itRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF166 contexts × 8 concurrency levels
    Peak decode
    186.7 tok/s
    Peak prefill
    9,728 tok/s
  • gemma-4-26B-A4B-itRTX PRO 6000 Blackwell Server Edition 96GB · sglang 0.5.22.dev236+g1093c501d · BF166 contexts × 8 concurrency levels
    Peak decode
    119.4 tok/s
    Peak prefill
    16.7K tok/s
  • liftRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,207.1 tok/s
    Peak prefill
    20.7K tok/s
  • liftRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF166 contexts × 8 concurrency levels
    Peak decode
    4,003.9 tok/s
    Peak prefill
    21.4K tok/s
  • North-Mini-Code-1.0RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF167 contexts × 8 concurrency levels
    Peak decode
    1,217.3 tok/s
    Peak prefill
    30.2K tok/s
  • Trinity-Nano-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    3,299.0 tok/s
    Peak prefill
    74K tok/s
  • Trinity-MiniRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,560.9 tok/s
    Peak prefill
    34.9K tok/s
  • Trinity-Mini-W4A16RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · INT45 contexts × 8 concurrency levels
    Peak decode
    3,105.1 tok/s
    Peak prefill
    34.8K tok/s
  • Trinity-Mini-NVFP4RTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.21.1rc1.dev305+gc8414a827 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    2,969.9 tok/s
    Peak prefill
    41K tok/s
  • Trinity-Mini-FP8-BlockRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · FP85 contexts × 8 concurrency levels
    Peak decode
    2,706.9 tok/s
    Peak prefill
    39.5K tok/s
  • Trinity-Mini-BaseRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,414.2 tok/s
    Peak prefill
    34.3K tok/s
  • Olmo-3.1-7B-RL-Zero-MathRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF164 contexts × 8 concurrency levels
    Peak decode
    687.1 tok/s
    Peak prefill
    23.2K tok/s
  • Olmo-3.1-7B-RL-Zero-CodeRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF164 contexts × 8 concurrency levels
    Peak decode
    686.4 tok/s
    Peak prefill
    23.1K tok/s
  • Olmo-3.1-32B-ThinkRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev452+g73a583112 · BF164 contexts × 8 concurrency levels
    Peak decode
    146.8 tok/s
    Peak prefill
    5,602 tok/s
  • Olmo-3.1-32B-InstructRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev151+g73a78e6f1 · BF164 contexts × 8 concurrency levels
    Peak decode
    146.6 tok/s
    Peak prefill
    5,618 tok/s
  • Olmo-3.1-32B-Instruct-SFTRTX PRO 6000 Blackwell Server Edition 96GB · vllm 0.30.1rc1.dev151+g73a78e6f1 · BF164 contexts × 8 concurrency levels
    Peak decode
    146.7 tok/s
    Peak prefill
    5,599 tok/s