ChartQA

Visual question answering over real-world chart images that require reading values off the chart and doing arithmetic or logical reasoning to answer.

Also known as: ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

unassessed

What it measures

ChartQA shows a model a chart image, drawn from real published sources, together with a natural-language question about it. The model must produce a short answer, which may require reading a value directly off the chart, comparing two or more values, or doing simple arithmetic or logical reasoning across several data points. Roughly three in ten question-answer pairs come from human annotators who wrote compositional, visually grounded questions; the rest were generated from human-written chart summaries and then checked. The task exercises visual perception of chart elements, such as axes, legends and bar heights, together with the reasoning needed to compute an answer, not just lookup.

Task format

Visual question answering over chart images; open-vocabulary short answers (numbers, phrases, yes/no)

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub