DocVQA

Question answering over scanned and typed document images, scored by fuzzy text match against reference answers.

Also known as: Document Visual Question Answering

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.
Categorymultimodal
Subcategorydocument understanding
Page statusactive
MetricANLS (Average Normalized Levenshtein Similarity)
Directionhigher_is_better
Unitscore (0-1)
Dataset size50000
PublisherComputer Vision Center (Universitat Autònoma de Barcelona) and IIIT Hyderabad, with Amazon

What it measures

DocVQA gives a model an image of a real document — a letter, memo, form, table or report — plus a natural-language question about its content, and asks for a short free-text answer. Questions require reading text in context (totals in a table, a form field's value, a handwritten note, a figure label) rather than isolated OCR, so the task combines text recognition, layout understanding and reading comprehension in a single English-language, single-image, single-turn call.

Task format

Document image plus a natural-language question in; a short free-text answer out.

Models reporting this benchmark

These figures come from the model cards, which carry one collection date per card and no per-score attribution. They are shown as reported, not as verified evidence.
ModelProviderScoreCard as of
Qwen3 VL 32B InstructAlibaba / Qwen Team96.52026-04
Qwen3 VL 8B InstructAlibaba / Qwen Team96.12026-04
Claude Sonnet 3.5Anthropic95.22026-04
Claude Sonnet 3.5 v2Anthropic95.22026-04
Qwen2 VL 7B InstructAlibaba / Qwen Team94.52025-03
Qwen2 VL 7B Instruct AWQAlibaba / Qwen Team94.52025-03
Llama 4 Maverick 17B 128E InstructMeta94.42026-04
Llama 4 Scout 17B 16EMeta94.42026-04
Llama 4 Scout 17B 16E InstructMeta94.42026-04
Llama-4-Maverick-17B-128E-Instruct-FP8Meta94.42026-04
Llama-4-Scout-17B-16E-Instruct-FP8Meta94.42026-04
GPT-4OpenAI94.12026-04
GPT-4.1OpenAI94.12026-04
GPT-4.1 miniOpenAI94.12026-04
GPT-4.1 nanoOpenAI94.12026-04
Gemini 2.5 ProGoogle DeepMind93.82026-04
Gemini 2.5 Pro Preview 05-06Google DeepMind93.82026-04
Gemini 2.5 Pro Preview 06-05Google DeepMind93.82026-04
Gemini 2.5 Pro Preview TTSGoogle DeepMind93.82026-04
Claude Opus 4Anthropic93.22026-04
Claude Opus 4.6Anthropic93.22026-04
MiniCPM V 4OpenBMB92.92026-04
MiniCPM V 4 5OpenBMB92.92026-04
MiniCPM V 4 5 ggufOpenBMB92.92026-04
GPT-4oOpenAI92.82026-04
GPT-4o (2024-05-13)OpenAI92.82026-04
GPT-4o (2024-08-06)OpenAI92.82026-04
GPT-4o (2024-11-20)OpenAI92.82026-04
GPT-4o miniOpenAI92.82026-04
NVLM D 72BNVIDIA92.62026-04
Claude Sonnet 4Anthropic91.52026-04
Claude Sonnet 4.5Anthropic91.52026-04
Claude Sonnet 4.5 (latest)Anthropic91.52026-04
Llama 3.2 90B VisionMeta90.12026-04
Llama 3.2 90B Vision InstructMeta90.12026-04
Qwen2.5-VL 72B InstructAlibaba / Qwen Team90.12026-04
Claude Opus 3Anthropic89.32026-04
Claude Sonnet 3Anthropic89.32026-04
Claude Haiku 3Anthropic88.82026-04
Gemini 1.5 ProGoogle DeepMind88.52026-04
Llama 3.2 11B VisionMeta88.42026-04
Llama 3.2 11B Vision InstructMeta88.42026-04
Gemini 2.0 FlashGoogle DeepMind88.12026-04
Gemini 2.0 Flash LiteGoogle DeepMind88.12026-04
GPT-4 TurboOpenAI87.22026-04
Gemma 3 12BGoogle DeepMind87.12026-04
Gemma 4 31BGoogle DeepMind86.82026-04
gemma 4 31B itGoogle DeepMind86.82026-04
gemma 4 31B it GGUFUnsloth86.82026-04
Gemma 4 31B IT NVFP4NVIDIA86.82026-04
Pixtral Large (latest)Mistral AI85.52026-04
Gemini 1.5 FlashGoogle DeepMind85.22026-04
Gemini 1.5 Flash-8BGoogle DeepMind85.22026-04
Qwen2.5 VL 3B InstructAlibaba / Qwen Team85.22026-04
Mistral Large (latest)Mistral AI83.52026-04
Mistral Large 3Mistral AI83.52026-04
Grok 2xAI82.52026-04
Grok 2 LatestxAI82.52026-04
Grok 2 VisionxAI82.52026-04
Grok 2 Vision (1212)xAI82.52026-04
Grok 2 Vision LatestxAI82.52026-04
Gemma 3 27BGoogle DeepMind82.12026-04
Command A VisionCohere81.22026-04
Pixtral 12BMistral AI78.82026-04
Qwen2.5-VL 7B InstructAlibaba / Qwen Team78.52026-04
Gemma 3 4BGoogle DeepMind75.82026-04
gemma 3 4B ptGoogle DeepMind75.82026-04

Data

This page as JSON · Edit on GitHub