CharXiv Reasoning (with tool use)

CharXiv Reasoning scored with the model given a tool during the eval -- an image-cropping tool in the one vendor report this page found -- instead of the static chart image alone.

Also known as: CharXiv-R w/ tools

unassessed

What it measures

This id captures CharXiv Reasoning scores produced when the model was allowed to use a tool while answering, rather than the base benchmark's default of a single static chart image and no tool access. Tool use of this kind is not part of CharXiv's own official protocol; it is an evaluation choice individual vendors make when they report the benchmark. The one primary source this page found that defines the tool explicitly is Anthropic's Claude Opus 4.6 system card (2026-02), which evaluated Claude Opus 4.6 and Claude Opus 4.5 with "a simple image-cropping tool" available -- letting the model zoom into a region of the chart before answering, rather than reading the whole image at its original resolution. The same system card documents reasoning models more generally (OpenAI's o3 and o4-mini, per their own system card) using tools including image cropping and Python-based analysis as part of their chain of thought, so cropping specifically targets the kind of fine print, dense legends and closely spaced data points that make CharXiv's charts hard to read at a glance.

Task format

The same open-ended, one-question-per-chart format as CharXiv Reasoning, except the model may invoke a tool -- in the documented case, cropping into the chart image -- before producing its final free-text answer.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub