MathVista

6,141 examples testing mathematical reasoning across charts, diagrams, word problems and textbook figures, pooled from 28 existing datasets plus three new ones.

unassessed

What it measures

MathVista tests whether a model can perform mathematical reasoning that depends on understanding a visual context, rather than reasoning over text alone. It spans five task types: figure question answering (reasoning over charts and plots), geometry problem solving, math word problems set in images, textbook question answering, and general visual question answering with a numeric or quantitative answer. Underlying visual contexts include natural images, geometry diagrams, abstract and synthetic scenes, function plots, and puzzle-test figures, and the questions draw on seven reasoning types from arithmetic and algebra to logical, statistical and scientific reasoning.

Task format

An image paired with a question, answered as either a multiple-choice letter or a free-form numerical answer (integer, one or two decimal places, or a list), in English.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub