SQuAD (2016) and SQuAD 2.0 (2018) are the canonical Wikipedia span-extraction reading sets; the official leaderboard has sat above human performance since 2019, so a modern score there is uninformative.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | extractive reading comprehension: locate an answer span in a given Wikipedia passage, or (SQuAD 2.0) determine that no answer is supported |
| Page status | saturated |
| Metric | Exact Match (EM) and F1 (word-overlap) against human reference answers, computed by the official evaluation script |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 98169 |
| Dataset licence | CC BY-SA 4.0 |
| Publisher | Stanford University |
SQuAD gives a model a paragraph from a Wikipedia article and a question about it, and the model must return the exact span of text in the paragraph that answers the question -- there is no free-form generation and, in the original 1.1 release, every question is guaranteed to be answerable from the given passage. The 2018 follow-up, SQuAD 2.0, adds a second requirement on top of the same task: over 50,000 questions written adversarially by crowdworkers to closely resemble answerable ones but have no answer in the passage, so a model must also decide when to abstain rather than guess a plausible-looking but wrong span. The two are frequently conflated under the single name "SQuAD," but they are meaningfully different tasks -- a system tuned only to extract spans has no mechanism for saying "no answer," and, as the 2.0 paper's own headline result shows, a strong system scoring 86% F1 on 1.1 drops to only 66% F1 once unanswerable questions are added.
Extractive span selection: given a passage and a question, output the start and end of the answer span within the passage verbatim (SQuAD 1.1), or the same plus an explicit "no answer" option (SQuAD 2.0). Modern LLM harnesses instead prompt the model to generate the answer text directly (or a literal "unanswerable" / "impossible to answer" string) rather than pick a span by position.
No model card in ModelSpec reports this benchmark yet.