Auto Categorization

A 328-item BIG-bench free-response task that names the shared category of a short list of words or objects.

Also known as: Autoclassification of Words/Objects

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	knowledge
Subcategory	BIG-bench free-response hypernym / category naming (328 items)
Page status	unknown
Metric	bleu
Direction	higher_is_better
Dataset size	328
Dataset licence	Apache-2.0
Publisher	Google (BIG-bench collaboration)

What it measures

auto_categorization gives a model a short English list of words, names or titles and asks it to name the broader class they belong to. Author Fanyue Xia built the items from Kaggle and Wikidata properties, grouping up to three examples per property under a varied sentence template. The intended skill is common-sense taxonomy, not a closed label set: the target is a word or phrase such as "colors" or "pets", and 95 items list more than one acceptable phrasing.

Task format

Free-response JSON task scored with BIG-bench BLEU. A canary GUID is embedded. There is no multiple-choice variant in the task directory.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub