NewsQA

NewsQA tests whether a model can answer questions with text spans from CNN news articles, including unanswerable questions.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	knowledge
Subcategory	extractive question answering
Page status	active
Metric	exact match over accepted reference strings (HELM)
Direction	higher_is_better
Unit	percent
Dataset size	12744
Publisher	Maluuba Research

What it measures

NewsQA presents a CNN article and a natural-language question written by a crowd worker. The answer is normally a span in the article, but some questions have no answer in context.

Task format

Passage, question, and answer prompt; extract a span or return No Answer.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub