Aan de slagLeaderboardDecode calculatorModellenOrganizationsReportsHardwareBenchmarksMarktplaatsVerhuurProAPI Docs
Taal
Actual Computer — Every computer, one endpoint

Benchmarks

Communitybenchmarksuites voor het evalueren van lokale LLM-kwaliteit. Dien resultaten in via de API.

Terminal-Bench 4.0Officieel
Pass rate across all runs · Question set · 66 questions

Terminal-Bench 4.0 (4.0.0) agentic terminal benchmark bundles. Task containers and verifier assets live in eval storage; Postgres stores only the manifest and pass/fail results.

Agentic12 runs
CRUDBench LocalMaxxing v2Officieel
Pass rate across all runs · Question set · 2 questions

Terminal-Bench 2.1 style CRUD API repair tasks, published through the Pro-gated terminal workflow.

Software engineering2 runs
CRUDbenchOfficieel
Pass rate across all runs · Question set · 40 questions

CRUDbench (0.4.1) agentic terminal benchmark bundles. Task containers and verifier assets live in eval storage; Postgres stores only the manifest and pass/fail results.

Agentic28 runs
Terminal-Bench 2.1Officieel
Pass rate across all runs · Question set · 89 questions

Terminal-Bench 2.1 (2.1) agentic terminal benchmark bundles. Task containers and verifier assets live in eval storage; Postgres stores only the manifest and pass/fail results.

Agentic149 runs
GSM8KOfficieel
Pass rate across all runs · Question set · 1,319 questions

GSM8K deterministic disjoint eval shards from openai/gsm8k/main:test. Running all 13 shards covers the full 1,319-question dataset; Postgres stores only shard metadata and pass/fail results.

Reasoning382 runs
HellaSwagOfficieel
Pass rate across all runs · Question set · 10,042 questions

HellaSwag deterministic disjoint eval shards from Rowan/hellaswag:validation. Running all 13 shards covers the full 10,042-question dataset; Postgres stores only shard metadata and pass/fail results.

Reasoning280 runs
ARC ChallengeOfficieel
Pass rate across all runs · Question set · 1,172 questions

ARC Challenge randomized eval shards from ai2_arc/ARC-Challenge:test. Question text lives in S3; Postgres stores only shard metadata and results.

Benchmark107 runs
HumanEval+Officieel
Pass rate across all runs · Question set · 164 questions

HumanEval+ randomized eval shards from evalplus/humanevalplus:test. Question text lives in S3; Postgres stores only shard metadata and results.

Coding123 runs
MBPP+Officieel
Pass rate across all runs · Question set · 378 questions

MBPP+ randomized eval shards from evalplus/mbppplus:test. Question text lives in S3; Postgres stores only shard metadata and results.

Coding62 runs