Get started Leaderboard Decode calculator Models Hardware Benchmarks Marketplace Rentals Pro API Docs

Language

Hardware

93 hardware groups · 156 total

RTX 3090

discrete gpu

774 benchmarks total43 users

Median56.2 tok/s

View speed tests →

10x variant1 runs

Median22.1 tok/s

8x variant18 runs

Median193 tok/s

7x variant2 runs

Median42.3 tok/s

6x variant1 runs

Median47.4 tok/s

4x variant13 runs

Median51.3 tok/s

3x variant1 runs

Median231 tok/s

2x variant109 runs

Median71.0 tok/s

Single GPU (1x)629 runs

Median55.1 tok/s

RTX 5090

discrete gpu

306 benchmarks total25 users

Median136 tok/s

View speed tests →

4x variant3 runs

Median157 tok/s

2x variant7 runs

Median138 tok/s

Single GPU (1x)296 runs

Median135 tok/s

GB10 Grace Blackwell

unified

183 benchmarks total22 users

Median53.0 tok/s

View speed tests →

RTX PRO 6000 Blackwell

discrete gpu

201 benchmarks total21 users

Median98.0 tok/s

View speed tests →

8x variant2 runs

Median74.3 tok/s

4x variant37 runs

Median146 tok/s

2x variant18 runs

Median238 tok/s

Single GPU (1x)144 runs

Median89.3 tok/s

Intel Arc Pro B70

discrete gpu

530 benchmarks total15 users

Median64.0 tok/s

View speed tests →

4x variant175 runs

Median47.6 tok/s

3x variant22 runs

Median45.8 tok/s

2x variant31 runs

Median58.5 tok/s

Single GPU (1x)302 runs

Median75.9 tok/s

Ryzen AI Max 395

unified

437 benchmarks total12 users

Median41.5 tok/s

View speed tests →

RTX 3060

discrete gpu

1120 benchmarks total11 users

Median52.4 tok/s

View speed tests →

2x variant558 runs

Median39.7 tok/s

Single GPU (1x)562 runs

Median65.2 tok/s

RX 7900 XTX

discrete gpu

122 benchmarks total11 users

Median66.4 tok/s

View speed tests →

2x variant4 runs

Median75.3 tok/s

Single GPU (1x)118 runs

Median66.4 tok/s

RTX 4090

discrete gpu

23 benchmarks total11 users

Median61.3 tok/s

View speed tests →

Radeon AI Pro R9700

discrete gpu

365 benchmarks total9 users

Median48.2 tok/s

View speed tests →

4x variant8 runs

Median232 tok/s

3x variant265 runs

Median39.3 tok/s

2x variant14 runs

Median84.0 tok/s

Single GPU (1x)78 runs

Median118 tok/s

RTX 5060 Ti

discrete gpu

25 benchmarks total9 users

Median69.3 tok/s

View speed tests →

2x variant12 runs

Median47.6 tok/s

Single GPU (1x)13 runs

Median109 tok/s

M5 Max

unified

40 benchmarks total6 users

Median88.3 tok/s

View speed tests →

M5 Pro

unified

39 benchmarks total5 users

Median67.2 tok/s

View speed tests →

Tesla P100

discrete gpu

32 benchmarks total4 users

Median44.2 tok/s

View speed tests →

3x variant1 runs

Median42.0 tok/s

2x variant27 runs

Median45.2 tok/s

Single GPU (1x)4 runs

Median28.3 tok/s

RTX 5070 Ti

discrete gpu

6 benchmarks total4 users

Median221 tok/s

View speed tests →

2x variant1 runs

Median141 tok/s

Single GPU (1x)5 runs

Median301 tok/s

RTX 4070

discrete gpu

181 benchmarks total3 users

Median37.8 tok/s

View speed tests →

RTX 3060 Ti

discrete gpu

163 benchmarks total3 users

Median71.7 tok/s

View speed tests →

RTX 3080

discrete gpu

47 benchmarks total3 users

Median71.2 tok/s

View speed tests →

Tesla V100

discrete gpu

36 benchmarks total3 users

Median37.7 tok/s

View speed tests →

2x variant4 runs

Median98.6 tok/s

Single GPU (1x)32 runs

Median33.8 tok/s

M3 Ultra

unified

28 benchmarks total3 users

Median39.4 tok/s

View speed tests →

M4 Max

unified

13 benchmarks total3 users

Median22.0 tok/s

View speed tests →

RTX 4060 Ti

discrete gpu

8 benchmarks total3 users

Median43.8 tok/s

View speed tests →

2x variant7 runs

Median43.6 tok/s

Single GPU (1x)1 runs

Median60.2 tok/s

Intel Arc Pro B60

discrete gpu

59 benchmarks total2 users

Median46.7 tok/s

View speed tests →

2x variant27 runs

Median44.5 tok/s

Single GPU (1x)32 runs

Median47.4 tok/s

GTX 1080 Ti

discrete gpu

27 benchmarks total2 users

Median34.3 tok/s

View speed tests →

RX 9070 XT

discrete gpu

27 benchmarks total2 users

Median567 tok/s

View speed tests →

M1 Max

unified

24 benchmarks total2 users

Median51.1 tok/s

View speed tests →

Strix Point Radeon 890M

unified

18 benchmarks total2 users

Median18.9 tok/s

View speed tests →

GTX 1060

discrete gpu

16 benchmarks total2 users

Median21.5 tok/s

View speed tests →

2x variant3 runs

Median18.3 tok/s

Single GPU (1x)13 runs

Median24.7 tok/s

RTX 5070

discrete gpu

13 benchmarks total2 users

Median64.5 tok/s

View speed tests →

M4 Pro

unified

12 benchmarks total2 users

Median48.4 tok/s

View speed tests →

RTX 5080

discrete gpu

12 benchmarks total2 users

Median145 tok/s

View speed tests →

RTX 3090 Ti

discrete gpu

9 benchmarks total2 users

Median119 tok/s

View speed tests →

2x variant2 runs

Median101 tok/s

Single GPU (1x)7 runs

Median119 tok/s

RTX 2080 Ti

discrete gpu

5 benchmarks total2 users

Median80.0 tok/s

View speed tests →

2x variant2 runs

Median20.5 tok/s

Single GPU (1x)3 runs

Median89.0 tok/s

Radeon 8060S

discrete gpu

2 benchmarks total2 users

Median16.9 tok/s

View speed tests →

RX 570 4GB

discrete gpu

224 benchmarks total1 user

Median44.0 tok/s

View speed tests →

2x variant174 runs

Median41.6 tok/s

Single GPU (1x)50 runs

Median103 tok/s

AMD Ryzen 7 7840HS

cpu only

87 benchmarks total1 user

Median219 tok/s

View speed tests →

2x RTX 3090 + RTX 3090 Ti

discrete gpu

25 benchmarks total1 user

Median72.1 tok/s

View speed tests →

3x variant25 runs

Median72.1 tok/s

Ryzen AI Max 385

unified

21 benchmarks total1 user

Median21.5 tok/s

View speed tests →

RX 6700 XT

discrete gpu

20 benchmarks total1 user

Median36.8 tok/s

View speed tests →

RTX A5000

discrete gpu

16 benchmarks total1 user

Median261 tok/s

View speed tests →

RTX 2080

discrete gpu

15 benchmarks total1 user

Median21.0 tok/s

View speed tests →

RX 7700 XT

discrete gpu

15 benchmarks total1 user

Median50.2 tok/s

View speed tests →

RTX 5000

discrete gpu

12 benchmarks total1 user

Median31.5 tok/s

View speed tests →

RX 5700 XT

discrete gpu

11 benchmarks total1 user

Median279 tok/s

View speed tests →

RX 9060 XT

discrete gpu

9 benchmarks total1 user

Median53.9 tok/s

View speed tests →

AMD Ryzen 7 8745H w/ Radeon 780M

cpu only

8 benchmarks total1 user

Median29.3 tok/s

View speed tests →

H200 NVL

discrete gpu

7 benchmarks total1 user

Median333 tok/s

View speed tests →

2x variant4 runs

Median333 tok/s

Single GPU (1x)3 runs

Median262 tok/s

H200 SXM

discrete gpu

7 benchmarks total1 user

Median496 tok/s

View speed tests →

4x variant7 runs

Median496 tok/s

RX 5700

discrete gpu

7 benchmarks total1 user

Median27.4 tok/s

View speed tests →

Ryzen 9 7940HS Radeon 780M Minisforum UM790 Pro

unified

7 benchmarks total1 user

Median19.5 tok/s

View speed tests →

RTX 3050

discrete gpu

6 benchmarks total1 user

Median43.5 tok/s

View speed tests →

RX 6700

discrete gpu

6 benchmarks total1 user

Median101 tok/s

View speed tests →

AMD Ryzen 7 8745H w/ Radeon 780M Graphics

cpu only

5 benchmarks total1 user

Median58.6 tok/s

View speed tests →

RTX 3060 + RTX 5060 Ti

discrete gpu

5 benchmarks total1 user

Median85.1 tok/s

View speed tests →

2x variant5 runs

Median85.1 tok/s

AMD Ryzen 7 9800X3D

cpu only

4 benchmarks total1 user

Best36.7k tok/s

Median11.5k tok/s

View speed tests →

Core Ultra Meteor Lake Ultra 7 155H

unified

4 benchmarks total1 user

Median10.3 tok/s

View speed tests →

Intel R Core TM Ultra 7 155H

cpu only

4 benchmarks total1 user

Median14.0 tok/s

View speed tests →

MT6897 TECNO POVA 7 Ultra 5G / Mali G615 MC6

unified

4 benchmarks total1 user

Median16.9 tok/s

View speed tests →

RTX A6000

discrete gpu

4 benchmarks total1 user

Median131 tok/s

View speed tests →

2x variant4 runs

Median131 tok/s

RX 6900 XT

discrete gpu

4 benchmarks total1 user

Median452 tok/s

View speed tests →

RX 7900 XT

discrete gpu

4 benchmarks total1 user

Median88.1 tok/s

View speed tests →

2x variant3 runs

Median116 tok/s

Single GPU (1x)1 runs

Median60.6 tok/s

M4 M4

unified

3 benchmarks total1 user

Median20.4 tok/s

View speed tests →

RTX 3090 + RTX 4090

discrete gpu

3 benchmarks total1 user

Median27.7 tok/s

View speed tests →

2x variant3 runs

Median27.7 tok/s

RX 9070

discrete gpu

3 benchmarks total1 user

Median47.0 tok/s

View speed tests →

Strix Halo Radeon 8060S

unified

3 benchmarks total1 user

Median105 tok/s

View speed tests →

A100

discrete gpu

2 benchmarks total1 user

Median37.4 tok/s

View speed tests →

AMD Ryzen 5 7640U

cpu only

2 benchmarks total1 user

Median25.4 tok/s

View speed tests →

Intel R Xeon R CPU E5 2699 v4 @ 2.20GHz

cpu only

2 benchmarks total1 user

Median12.3 tok/s

View speed tests →

Jetson Orin Orin Nano Super Developer Kit

unified

2 benchmarks total1 user

Median21.9 tok/s

View speed tests →

T4

discrete gpu

2 benchmarks total1 user

Median57.0 tok/s

View speed tests →

5060 Ti

discrete gpu

1 benchmark total1 user

Median23.0 tok/s

View speed tests →

AMD Ryzen Threadripper 9960X 24 Cores

cpu only

1 benchmark total1 user

Median52.2 tok/s

View speed tests →

GTX 1650

discrete gpu

1 benchmark total1 user

Median30.6 tok/s

View speed tests →

H100

discrete gpu

1 benchmark total1 user

Median1.4k tok/s

View speed tests →

M1 Pro

unified

1 benchmark total1 user

Median11.1 tok/s

View speed tests →

M2 Pro

unified

1 benchmark total1 user

Median33.0 tok/s

View speed tests →

M5 M5

unified

1 benchmark total1 user

Median34.3 tok/s

View speed tests →

Qualcomm Snapdragon 888 ARM64

cpu only

1 benchmark total1 user

Median6.2 tok/s

View speed tests →

Radeon AI Pro R9700 + RX 7900 XTX

discrete gpu

1 benchmark total1 user

Median36.7 tok/s

View speed tests →

2x variant1 runs

Median36.7 tok/s

RTX 3070 Ti

discrete gpu

1 benchmark total1 user

Median33.5 tok/s

View speed tests →

RTX 3080 Ti

discrete gpu

1 benchmark total1 user

Median15.4 tok/s

View speed tests →

RTX 3090 + RTX 4090 + RTX 5090 + RTX 5090

discrete gpu

1 benchmark total1 user

Median89.7 tok/s

View speed tests →

4x variant1 runs

Median89.7 tok/s

RTX 4060

discrete gpu

1 benchmark total1 user

Median29.9 tok/s

View speed tests →

RTX 4070 Super

discrete gpu

1 benchmark total1 user

Median77.4 tok/s

View speed tests →

RTX 4070 Ti + RTX 5080

discrete gpu

1 benchmark total1 user

Median428 tok/s

View speed tests →

2x variant1 runs

Median428 tok/s

RTX 5060 Ti + RTX 5070 Ti

discrete gpu

1 benchmark total1 user

Median34.3 tok/s

View speed tests →

2x variant1 runs

Median34.3 tok/s

RTX PRO 6000

discrete gpu

1 benchmark total1 user

Median92.8 tok/s

View speed tests →

2x variant1 runs

Median92.8 tok/s

RX 6600

discrete gpu

1 benchmark total1 user

Median40.5 tok/s

View speed tests →

2x variant1 runs

Median40.5 tok/s

RX 6750 XT

discrete gpu

1 benchmark total1 user

Median500 tok/s

View speed tests →

RX 6800

discrete gpu

1 benchmark total1 user

Median89.2 tok/s

View speed tests →

RX 6950 XT

discrete gpu

1 benchmark total1 user

Median222 tok/s

View speed tests →

Ryzen AI Max Max 395

unified

1 benchmark total1 user

Median12.7 tok/s

View speed tests →

Ryzen AI Max Radeon 780M

unified

1 benchmark total1 user

Median26.9 tok/s

View speed tests →