Suite di benchmark della community per valutare la qualità degli LLM locali. Invia i risultati tramite API.
OpenAI HumanEval via EleutherAI lm-evaluation-harness task humaneval, 0-shot, pass@k code-generation scoring.
Massive Multitask Language Understanding via EleutherAI lm-evaluation-harness task mmlu, 5-shot, exact-match/accuracy style scoring.
