{
 "body": "## What it measures\n\nDr. DocBench evaluates document parsing and recognition on expert-level pages that defeat easy OCR and layout tests. Its multilingual book corpus spans 52 BISAC subject domains and includes difficult structures such as chemical formulae, music notation, complex tables, and cross-page layouts.\n\nThe benchmark provides page- and block-level annotations for layout, reading order, hierarchical relations, and domain-specific visual content. It targets both document pipelines and general vision-language models.\n\n## How it is scored\n\nThe paper evaluates parsing quality over the annotated structures and compares pipeline parsers with general-purpose VLMs. The abstract does not define one universal metric name, maximum, or human baseline. Report the annotation type and parser configuration with every score, because layout, reading order, and domain content measure different capabilities.\n\n## Dataset and licence\n\nThe paper reports 4,514 annotated pages from long documents averaging around 100 pages, with approximately 65,000 high-quality page- and block-level annotations. It describes a parser-failure-based sampling process from a large multilingual book corpus. The consulted source does not establish the dataset licence or answer visibility, so those fields remain unknown.\n\n## Who publishes it\n\nDr. DocBench was introduced by Minglai Yang and 24 coauthors in a May 2026 arXiv paper. The arXiv record is the primary source consulted. No public leaderboard is established in the abstract.\n\n## Lineage\n\nDr. DocBench is a standalone expert-level document parsing benchmark. It responds to limitations in common OCR and parsing benchmarks but does not identify a single predecessor or successor page.\n\n## Saturation and contamination\n\nThe authors report that strong performance on existing benchmarks does not transfer to Dr. DocBench and that failures remain across subjects, content types, and structural attributes. The benchmark is open. Training exposure is not established by the paper, so contamination risk is unknown.\n\n## How to run it\n\nUse the released pages and annotations with the paper\u2019s parser evaluation. Report language, document domain, page or block task, OCR preprocessing, layout model, and any VLM prompt. Preserve long-document context where the task requires cross-page relations.\n\n## Reading the numbers\n\nA high score means a parser recovered the selected document structures on difficult pages. It does not guarantee reliable extraction from every domain or document format. Inspect performance by subject and structure, since aggregate quality can hide failures in tables or reading order. Compare against ordinary OCR benchmarks only as a difficulty contrast.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "dr_docbench",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "category": "multimodal",
  "contamination": {
   "note": "The paper does not establish training-data exposure.",
   "risk": "unknown"
  },
  "dataset": {
   "languages": [
    "multilingual"
   ],
   "modalities": [
    "image",
    "text"
   ],
   "public_test_set": null,
   "size": 4514,
   "size_note": "4,514 annotated pages and approximately 65,000 page- and block-level annotations."
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna, luna-stream-a-006 (Codex coordinated)",
   "reviewed": "",
   "reviewed_by": ""
  },
  "harness": {
   "other": "Dr. DocBench evaluation implementation described by the paper."
  },
  "id": "dr_docbench",
  "measures": "Dr. DocBench tests vision-language models and document parsers on challenging pages from long multilingual books. It spans 52 BISAC subject domains and targets chemical formulae, music notation, complex tables, cross-page layouts, and other structures where modern parsers fail.",
  "metric": {
   "direction": "higher_is_better",
   "name": "parsing quality",
   "unit": "score"
  },
  "name": "Dr. DocBench",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2606.01393",
   "title": "Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing",
   "url": "https://arxiv.org/abs/2606.01393",
   "year": 2026
  },
  "publisher": {
   "authors": [
    "Minglai Yang",
    "Xinyan Velocity Yu",
    "Pengyuan Li",
    "Xinyu Guo",
    "Zhenting Qi",
    "Konwoo Kim",
    "Longtian Ye",
    "Xiaolong Luo",
    "Jinhe Bi",
    "Henry Zhang",
    "Haris Riaz",
    "Xuan Zhang",
    "Yunze Xiao",
    "Bangya Liu",
    "Tom Tang",
    "Yunfei Zhao",
    "Qunshu Lin",
    "Zihan Wang",
    "Minghao Liu",
    "Michael Lingzhi Li",
    "Yilun Du",
    "Jesse Thomason",
    "Rogerio Feris",
    "Alex Pentland",
    "Zexue He"
   ],
   "org": "Dr. DocBench authors",
   "url": "https://arxiv.org/abs/2606.01393"
  },
  "released": "2026-05",
  "saturation": {
   "note": "The paper reports substantial failures by strong existing parsers and VLMs.",
   "status": "open"
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "Dr. DocBench paper",
    "url": "https://arxiv.org/abs/2606.01393"
   }
  ],
  "subcategory": "document parsing",
  "summary": "Dr. DocBench evaluates expert-level document parsing on difficult multilingual pages with layout, reading-order, and domain-specific annotations.",
  "tags": [
   "documents",
   "OCR",
   "layout",
   "vision-language"
  ],
  "task_format": "Document-page parsing with page- and block-level structural annotations."
 }
}