ModelsLeaderboardHardwareEvalsTrainRentalsAPI Docs
Language

Hardware leaderboard

See which hardware appears most in approved benchmarks, then drill in by hardware type.

AMD Radeon AI Pro R9700

108 runs

Best
1.4k tok/s
Median
67.5 tok/s
Min
6.2 tok/s
3x variant93 runs2x variant10 runs

Strix Halo Ryzen AI MAX 395 Radeon 8060S

90 runs

Best
107 tok/s
Median
53.1 tok/s
Min
5.2 tok/s

RTX 3090

84 runs

Best
241 tok/s
Median
38.6 tok/s
Min
5.5 tok/s
8x variant3 runs4x variant1 runs2x variant37 runs

Intel Arc Pro B70 32GB

79 runs

Best
89.3 tok/s
Median
45.2 tok/s
Min
13.8 tok/s
4x variant52 runs3x variant15 runs2x variant9 runs

Ryzen AI Max 395

68 runs

Best
82.6 tok/s
Median
38.0 tok/s
Min
9.4 tok/s

Intel Arc Pro B70

68 runs

Best
70.3 tok/s
Median
33.7 tok/s
Min
0.5 tok/s
4x variant48 runs3x variant7 runs2x variant2 runs

RTX 3080

37 runs

Best
149 tok/s
Median
74.0 tok/s
Min
10.0 tok/s

GB10 Blackwell DGX Spark

29 runs

Best
102 tok/s
Median
31.0 tok/s
Min
4.7 tok/s

RX 7900 XTX

27 runs

Best
577 tok/s
Median
136 tok/s
Min
43.6 tok/s

GTX 1080 Ti

26 runs

Best
94.9 tok/s
Median
30.9 tok/s
Min
2.2 tok/s

NVIDIA RTX PRO 6000 Blackwell Workstation Edition

25 runs

Best
170 tok/s
Median
83.0 tok/s
Min
14.7 tok/s

RTX3060+RTX2080-pooled-20GB

25 runs

Best
150 tok/s
Median
74.0 tok/s
Min
35.0 tok/s

NVIDIA GeForce RTX 3090

22 runs

Best
164 tok/s
Median
49.6 tok/s
Min
7.4 tok/s
8x variant2 runs4x variant6 runs2x variant8 runs

RTX 5090

18 runs

Best
241 tok/s
Median
104 tok/s
Min
55.4 tok/s

RTX 3060

17 runs

Best
149 tok/s
Median
69.0 tok/s
Min
35.0 tok/s

NVIDIA RTX A5000

16 runs

Best
325 tok/s
Median
261 tok/s
Min
32.3 tok/s

M3 M3 Ultra

15 runs

Best
141 tok/s
Median
57.1 tok/s
Min
18.5 tok/s

RTX 2080

15 runs

Best
93.0 tok/s
Median
21.0 tok/s
Min
10.0 tok/s

RTX 4090

14 runs

Best
214 tok/s
Median
55.3 tok/s
Min
41.8 tok/s

M5 M5 Pro

13 runs

Best
106 tok/s
Median
76.5 tok/s
Min
5.4 tok/s

AMD Radeon RX 7900 XTX

12 runs

Best
105 tok/s
Median
60.0 tok/s
Min
28.5 tok/s

M5 M5 Max

12 runs

Best
122 tok/s
Median
53.0 tok/s
Min
17.8 tok/s

Strix Halo Ryzen AI Max 395

11 runs

Best
56.8 tok/s
Median
26.3 tok/s
Min
3.8 tok/s

Tesla P100-PCIE-16GB

10 runs

Best
144 tok/s
Median
41.1 tok/s
Min
7.5 tok/s
2x variant6 runs

M4 Max

9 runs

Best
135 tok/s
Median
31.0 tok/s
Min
13.0 tok/s

AMD Radeon RX 9070 XT

9 runs

Best
96.6 tok/s
Median
24.9 tok/s
Min
1.0 tok/s

NVIDIA GeForce RTX 5090

8 runs

Best
286 tok/s
Median
157 tok/s
Min
54.6 tok/s
2x variant2 runs

NVIDIA RTX PRO 6000 Blackwell

7 runs

Best
506 tok/s
Median
172 tok/s
Min
74.0 tok/s
8x variant2 runs

NVIDIA H200 NVL

7 runs

Best
2.7k tok/s
Median
333 tok/s
Min
175 tok/s
2x variant4 runs

Ryzen 9 7940HS Radeon 780M Minisforum UM790 Pro

7 runs

Best
24.8 tok/s
Median
19.5 tok/s
Min
2.8 tok/s

NVIDIA H200 SXM

7 runs

Best
878 tok/s
Median
496 tok/s
Min
197 tok/s
4x variant7 runs

Radeon RX 7900 XTX

6 runs

Best
64.1 tok/s
Median
28.4 tok/s
Min
1.6 tok/s

Blackwell GB10

5 runs

Best
90.0 tok/s
Median
30.0 tok/s
Min
8.0 tok/s

RTX 3090 Ti

5 runs

Best
140 tok/s
Median
132 tok/s
Min
70.3 tok/s
2x variant2 runs

Strix Halo Ryzen AI Max+ 395

5 runs

Best
17.0 tok/s
Median
12.4 tok/s
Min
7.0 tok/s

NVIDIA GeForce RTX 3090 Ti

4 runs

Best
156 tok/s
Median
36.0 tok/s
Min
32.6 tok/s

Multi-GPU

4 runs

Best
34.3 tok/s
Median
30.8 tok/s
Min
24.0 tok/s
2x variant4 runs

M4 M4 Max

4 runs

Best
83.4 tok/s
Median
20.1 tok/s
Min
18.4 tok/s

NVIDIA GeForce RTX 4070

4 runs

Best
62.1 tok/s
Median
57.4 tok/s
Min
54.5 tok/s

Intel(R) Core(TM) Ultra 7 155H

4 runs

Best
27.6 tok/s
Median
14.0 tok/s
Min
8.4 tok/s

Core Ultra (Meteor Lake) Ultra 7 155H

4 runs

Best
28.0 tok/s
Median
10.3 tok/s
Min
6.5 tok/s

AMD RX 9060 XT 16GB

4 runs

Best
66.8 tok/s
Median
59.8 tok/s
Min
39.8 tok/s

NVIDIA GB10 Grace Blackwell (DGX Spark)

4 runs

Best
27.8 tok/s
Median
27.0 tok/s
Min
26.6 tok/s
2x variant4 runs

NVIDIA GB10 Grace Blackwell (DGX Spark, single)

4 runs

Best
78.9 tok/s
Median
32.4 tok/s
Min
27.7 tok/s

MT6897 TECNO POVA 7 Ultra 5G / Mali-G615 MC6

4 runs

Best
19.1 tok/s
Median
16.9 tok/s
Min
7.1 tok/s

RTX 3060 + RTX 2080

4 runs

Best
161 tok/s
Median
83.5 tok/s
Min
37.0 tok/s

RTX 2080 Ti

3 runs

Best
106 tok/s
Median
89.0 tok/s
Min
80.0 tok/s

AMD Radeon RX 9070

3 runs

Best
76.0 tok/s
Median
47.0 tok/s
Min
34.2 tok/s

AMD RX 9060 XT

3 runs

Best
100 tok/s
Median
90.6 tok/s
Min
38.1 tok/s

Strix Halo Radeon 8060S Graphics

3 runs

Best
256 tok/s
Median
105 tok/s
Min
88.3 tok/s

RTX 5060 Ti

2 runs

Best
60.9 tok/s
Median
46.5 tok/s
Min
32.1 tok/s
2x variant2 runs

NVIDIA RTX A6000

2 runs

Best
133 tok/s
Median
115 tok/s
Min
97.3 tok/s
2x variant2 runs

Jetson Orin Orin Nano Super Developer Kit

2 runs

Best
27.9 tok/s
Median
21.9 tok/s
Min
15.9 tok/s

NVIDIA GeForce RTX 4090

2 runs

Best
165 tok/s
Median
126 tok/s
Min
87.7 tok/s

RTX A6000

2 runs

Best
166 tok/s
Median
147 tok/s
Min
129 tok/s
2x variant2 runs

NVIDIA GeForce RTX 5070 Ti

2 runs

Best
141 tok/s
Median
137 tok/s
Min
133 tok/s
2x variant1 runs

NVIDIA DGX Spark GB10 Grace Blackwell sm_121a 2x nodes

2 runs

Best
20.4 tok/s
Median
16.2 tok/s
Min
11.9 tok/s

RTX PRO 6000

1 runs

Best
92.8 tok/s
Median
92.8 tok/s
Min
92.8 tok/s
2x variant1 runs

5060 Ti

1 runs

Best
23.0 tok/s
Median
23.0 tok/s
Min
23.0 tok/s

NVIDIA GeForce RTX 4060 Ti 16GB

1 runs

Best
60.2 tok/s
Median
60.2 tok/s
Min
60.2 tok/s

NVIDIA GeForce RTX 3070 Ti

1 runs

Best
33.5 tok/s
Median
33.5 tok/s
Min
33.5 tok/s

M5 Pro

1 runs

Best
105 tok/s
Median
105 tok/s
Min
105 tok/s

GTX 1650

1 runs

Best
30.6 tok/s
Median
30.6 tok/s
Min
30.6 tok/s

GB10 DGX Spark GB10

1 runs

Best
26.9 tok/s
Median
26.9 tok/s
Min
26.9 tok/s

RTX 5070 Ti

1 runs

Best
124 tok/s
Median
124 tok/s
Min
124 tok/s

RTX 4060 Ti 16GB

1 runs

Best
62.0 tok/s
Median
62.0 tok/s
Min
62.0 tok/s
2x variant1 runs

RTX 4070 SUPER

1 runs

Best
77.4 tok/s
Median
77.4 tok/s
Min
77.4 tok/s

M2 M2 Pro

1 runs

Best
33.0 tok/s
Median
33.0 tok/s
Min
33.0 tok/s

AMD Radeon RX 6800

1 runs

Best
89.2 tok/s
Median
89.2 tok/s
Min
89.2 tok/s

Ryzen AI Max Max+ 395

1 runs

Best
12.7 tok/s
Median
12.7 tok/s
Min
12.7 tok/s

NVIDIA GeForce RTX 5080

1 runs

Best
151 tok/s
Median
151 tok/s
Min
151 tok/s

AMD Radeon 8060S Graphics (Strix Halo APU, gfx1151)

1 runs

Best
14.8 tok/s
Median
14.8 tok/s
Min
14.8 tok/s

Qualcomm Snapdragon 888 ARM64

1 runs

Best
6.2 tok/s
Median
6.2 tok/s
Min
6.2 tok/s

NVIDIA GeForce RTX 3060

1 runs

Best
35.0 tok/s
Median
35.0 tok/s
Min
35.0 tok/s

RTX 3080 Ti

1 runs

Best
15.4 tok/s
Median
15.4 tok/s
Min
15.4 tok/s

GB10 DGX Spark

1 runs

Best
17.7 tok/s
Median
17.7 tok/s
Min
17.7 tok/s

GTX 1060 6GB

1 runs

Best
15.0 tok/s
Median
15.0 tok/s
Min
15.0 tok/s

Tesla V100 SXM2 32GB

1 runs

Best
66.8 tok/s
Median
66.8 tok/s
Min
66.8 tok/s

GB10 Gigabyte AI TOP ATOM/Spark

1 runs

Best
62.1 tok/s
Median
62.1 tok/s
Min
62.1 tok/s

GB10 Grace Blackwell

1 runs

Best
39.7 tok/s
Median
39.7 tok/s
Min
39.7 tok/s

NVIDIA DGX Spark GB10 (Grace Blackwell, sm_121a)

1 runs

Best
94.8 tok/s
Median
94.8 tok/s
Min
94.8 tok/s

AMD Radeon RX 9060 XT 16GB

1 runs

Best
53.9 tok/s
Median
53.9 tok/s
Min
53.9 tok/s

RX 9060 XT

1 runs

Best
52.5 tok/s
Median
52.5 tok/s
Min
52.5 tok/s

Ryzen AI Max Radeon 780M

1 runs

Best
26.9 tok/s
Median
26.9 tok/s
Min
26.9 tok/s

RTX PRO 6000 Blackwell Server Edition

1 runs

Best
128 tok/s
Median
128 tok/s
Min
128 tok/s

NVIDIA GB10 Grace Blackwell (Dual DGX Spark, RPC)

1 runs

Best
16.1 tok/s
Median
16.1 tok/s
Min
16.1 tok/s
2x variant1 runs

NVIDIA GeForce RTX 4060

1 runs

Best
29.9 tok/s
Median
29.9 tok/s
Min
29.9 tok/s

RTX PRO 6000 Blackwell

1 runs

Best
1.0k tok/s
Median
1.0k tok/s
Min
1.0k tok/s

NVIDIA GeForce RTX 5060 Ti

1 runs

Best
109 tok/s
Median
109 tok/s
Min
109 tok/s

NVIDIA GeForce RTX 5060 Ti 16GB

1 runs

Best
158 tok/s
Median
158 tok/s
Min
158 tok/s

NVIDIA GeForce RTX 5070

1 runs

Best
64.5 tok/s
Median
64.5 tok/s
Min
64.5 tok/s

AMD Radeon RX 5700 XT

1 runs

Best
61.6 tok/s
Median
61.6 tok/s
Min
61.6 tok/s

AMD Radeon RX 6950 XT

1 runs

Best
222 tok/s
Median
222 tok/s
Min
222 tok/s