ASCII MNIST

A BIG-bench task that asks a model to identify MNIST digits rendered as ASCII art.

Also known as: BIG-bench mnist_ascii

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	multimodal
Subcategory	ASCII-art digit recognition
Page status	unknown
Metric	multiple_choice_grade
Direction	higher_is_better
Unit	%
Dataset size	69984
Dataset licence	Apache-2.0
Publisher	Google BIG-bench

What it measures

The task converts MNIST digit images into ASCII art and asks which digit is shown. The input is text arranged as a visual pattern, so the task probes whether a language model can switch from language processing to simple visual layout reading.

Task format

Ten-way multiple choice over digit labels 0 through 9; BIG-bench preferred metric is multiple_choice_grade.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub