113k four-way multiple-choice questions asking which of four captions plausibly continues a video-derived situation; HellaSwag's direct predecessor.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | grounded commonsense inference: predicting the next event in a video-caption situation |
| Page status | saturated |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 113000 |
| Dataset licence | MIT (per the rowanz/swagaf GitHub repository's LICENSE file); the Hugging Face dataset card lists licence as unknown |
| Publisher | Paul G. Allen School of Computer Science & Engineering, University of Washington |
SWAG tests grounded commonsense inference: given a sentence describing part of a real-world situation (drawn from video captioning corpora), a model must pick which of four candidate sentences most plausibly follows. One ending is the true next caption; the other three are adversarially selected machine-generated distractors designed to look plausible on the surface while being wrong. The source situations come from ActivityNet Captions (short YouTube activity clips) and the Large Scale Movie Description Challenge (LSMDC, movie audio-description captions), so the task is English text only, though it is grounded in described physical and social activity rather than abstract text.
Four-way multiple choice: given a context sentence, select the most plausible of four candidate continuations.
No model card in ModelSpec reports this benchmark yet.