OCRBench

1,000 hand-verified questions across five OCR task types, testing whether a multimodal model can read and reason about text embedded in images.

unassessed

What it measures

OCRBench evaluates optical character recognition ability in large multimodal models across five task types: plain text recognition, scene-text-centric visual question answering, document-oriented visual question answering, key information extraction, and handwritten mathematical expression recognition. Items are drawn from 29 existing OCR-related datasets and re-verified by hand into 1,000 question-answer pairs, so the benchmark is broader than any single source dataset while staying small enough to evaluate cheaply.

Task format

Open-ended, short-answer visual question answering over an image containing text; the model must read the text to answer.

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub