OpenAI's 220-task gold set of real occupation work products; models produce files that experts (or OpenAI's grader) compare with human deliverables.
unassessed
| Category | agentic |
|---|---|
| Subcategory | occupation deliverables across top U.S. GDP sectors |
| Page status | active |
| Metric | win rate versus human expert deliverable |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 220 |
| Publisher | OpenAI |
GDPval asks whether a model can finish economically valuable knowledge-work tasks the way an experienced professional would. Tasks cover 44 occupations in the nine U.S. sectors that contribute most to GDP, mapped to O*NET work activities. Each item is a prompt plus reference files, based on real work product from experts with a stated average of 14 years' experience (minimum four). Deliverables include briefs, spreadsheets, slides, and similar files, not a lettered exam answer. Inspect Evals and the public Hugging Face repo ship only the 220-task gold subset (five tasks per occupation). The paper's full set is 1,320 tasks (at least 30 per occupation) and is not in the open download.
Agent run in Docker with bash and python (180s). The model writes files under deliverable_files/. Inspect's built-in scorer is exact() only so a Score object exists; official quality is pairwise comparison against a human gold deliverable, via experts or OpenAI's free auto-grader after a Hugging Face upload.
No model card in ModelSpec reports this benchmark yet.