Suites de benchmarks communautaires pour évaluer la qualité des LLM locaux. Envoyez les résultats via l'API.
Community-scored creative writing eval for short tech-related 4chan-style greenposts. Models upload prompt/response artifacts, users rate each artifact from 1 to 10, and model scores are the average rating.
OpenAI HumanEval via EleutherAI lm-evaluation-harness task humaneval, 0-shot, pass@k code-generation scoring.
Massive Multitask Language Understanding via EleutherAI lm-evaluation-harness task mmlu, 5-shot, exact-match/accuracy style scoring.
