Winogrande African Languages

HELM's winogrande_afr scenario evaluates commonsense pronoun resolution using Winogrande items translated into 11 low-resource African languages.

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	knowledge
Subcategory	reasoning
Page status	active
Metric	exact match
Direction	higher_is_better
Unit	percent
Publisher	Stanford CRFM (HELM integration); translations from the Bridging the Gap project

What it measures

HELM's winogrande_afr scenario evaluates commonsense pronoun resolution using Winogrande-style multiple-choice items translated into 11 low-resource African languages, not English.

Task format

Multiple-choice pronoun resolution; each item is a sentence with an ambiguous pronoun and two candidate referents, scored by exact match on the correct choice.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub