Discourse Marker Prediction

An 857-item BIG-bench 10-way task that asks which English discourse marker continues a sentence pair, drawn from Discovery.

Also known as: BIG-bench discourse_marker_prediction

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categoryknowledge
SubcategoryBIG-bench 10-way discourse connective prediction (857 items)
Page statusunknown
Metricmultiple_choice_grade
Directionhigher_is_better
Unit%
Dataset size857
Dataset licenceApache-2.0
PublisherGoogle (BIG-bench collaboration)

What it measures

discourse_marker_prediction gives one English sentence and ten candidate continuations that differ mainly in the opening discourse marker. The model must pick the continuation whose marker is the original connective. Author Damien Sileo filtered Discovery test pairs so a supervised RoBERTa was both correct and at least 10% more confident than the runner-up, then sampled about five such items per marker. The skill is rare connective choice, not PDTB implicit-relation labelling. It is not [bbh](bbh.md).

Task format

Ten-option multiple choice with preferred metric multiple_choice_grade. append_choices_to_input is true. Keywords include zero-shot, one-shot and many-shot. Canary GUID embedded.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub