MBPP

Official

Mostly Basic Python Problems — 500 crowd-sourced Python programming problems with automated test cases. Broader coverage than HumanEval.

Category: codingRunner: lm-eval-harnessVersion: v1.0Submitted by: Community

Eval Details

Scoring

Pass At K

Aggregation

Mean

Direction

Higher is better

Tasks

1 task

Seed: 42FewShot: 3

Task	Dataset	Weight	Shots	Max Tokens
MBPP (pass@1) mbpp	google-research-datasets/mbpp / sanitized / test	1	3-shot	—

No approved results yet. Submit a run via the API.