648 questions built from 50 Wall Street Journal articles that ask a model to name the implicit antecedent behind a bridging phrase, such as "limited access of what?", using only the preceding context.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | indirect (bridging) anaphora resolution recast as extractive question answering |
| Page status | unknown |
| Metric | exact_str_match |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 648 |
| Dataset licence | Apache-2.0 for the BIG-bench repository and task code, confirmed via GitHub's own licence detection; the underlying text is drawn from Wall Street Journal articles via the ISNotes corpus, whose own redistribution terms were not independently confirmed during this research |
This task casts bridging anaphora resolution as question answering. Anaphora is ordinary back-reference ("she," "it"), but bridging (or indirect) anaphora is subtler: a noun phrase like "limited access" or "the manager" refers back to something only implied by, not identical to, an earlier expression, connected through lexical, frame or world knowledge rather than shared identity. The task's own example: given a passage describing colour-coded post-earthquake building inspections, the phrase "limited access" bridges back to "buildings with substantial damage," a connection a model must recover from the discourse as a whole, not from local syntax. Converting this into question answering (asking, for instance, "limited access of what?") removes the gold-standard candidate-mention list that older pairwise bridging-resolution models were given for free, so a model has to locate the antecedent in open text rather than rank a provided shortlist -- a harder and more realistic setting the task's own documentation argues is closer to how the skill would actually be used.
"Context: <all sentences up to and including the one containing the anaphor> Question: <bridging question, e.g. 'limited access of what?'> Answer: <antecedent noun phrase>" -- free-response extraction, zero-shot, no multiple-choice options. Roughly 26% of anaphors have their antecedent in the same sentence, and 23% have one more than two sentences away, per the task's own documentation, so the task cannot be solved by only checking the immediately preceding sentence.
No model card in ModelSpec reports this benchmark yet.