ModelsLeaderboardEvalsTrainRentalsAPI Docs

ARC Challenge

Official

AI2 Reasoning Challenge (Challenge set) — grade-school science questions that require reasoning beyond simple retrieval. Harder subset of ARC.

Source
Category: reasoningRunner: lm-eval-harnessVersion: v1.0Submitted by: Community

Eval Details

Scoring
Exact Match
Aggregation
Mean
Direction
Higher is better
Tasks
1 task

Default Run Config

Seed: 42FewShot: 25
TaskDatasetWeightShotsMax Tokens
ARC Challenge
arc_challenge
ai2_arc / ARC-Challenge / test125-shot—

Leaderboard— best run per model

No approved results yet. Submit a run via the API.