{
 "body": "## What it measures\n\nMRAG is a biomedical retrieval-augmented generation benchmark for English and Chinese tasks. It uses a corpus built from Wikipedia and PubMed and evaluates systems that retrieve evidence before generating an answer.\n\nThe benchmark is intended to measure RAG reliability and usefulness in scientific and clinical question answering. It also examines reasoning quality and readability for long-form responses.\n\n## How it is scored\n\nThe paper reports task performance under different retrieval approaches, model sizes, and prompting strategies. It finds that retrieval improves reliability and usefulness while responses can become slightly less readable on long-form questions. The abstract does not establish one metric name, maximum, or human baseline, so these remain unknown. Reproduction should use MRAG-Toolkit and report task and language separately.\n\n## Dataset and licence\n\nThe corpus uses Wikipedia and PubMed material and covers English and Chinese. The authors state that MRAG-Bench\u2019s dataset and toolkit will be released under CC BY 4.0 upon acceptance. Because that statement is future-oriented in the paper version consulted, release status and exact split sizes remain unresolved.\n\n## Who publishes it\n\nLiz Li and Wei Zhu introduced MRAG in a 2026 arXiv paper. The paper names the MRAG-Toolkit for systematic RAG experiments. No current public leaderboard is established.\n\n## Lineage\n\nMRAG is a biomedical RAG benchmark motivated by the lack of comprehensive medical-domain evaluation. It is distinct from MRAG-Bench, which evaluates vision-centric multimodal retrieval. The paper does not name a successor.\n\n## Saturation and contamination\n\nThe reported retrieval, model-size, and prompting effects show an open evaluation space. Wikipedia and PubMed are public sources, so training exposure is plausible, but the paper does not establish model-specific contamination. Risk is unknown.\n\n## How to run it\n\nUse the MRAG-Toolkit when released, select the language and task, and report corpus snapshot, retriever, top-k evidence, prompt, generator, and scoring method. Keep retrieval ablations and long-form readability analysis separate from answer correctness.\n\n## Reading the numbers\n\nA high score indicates good performance for the selected biomedical task and retrieval configuration. It does not establish clinical safety or evidence faithfulness. Compare English and Chinese results, retrieval ablations, and long-form readability. Inspect citations and unsupported claims before deployment.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "mrag",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "category": "domain",
  "contamination": {
   "note": "The corpus draws on public Wikipedia and PubMed material, but model-specific exposure is not established.",
   "risk": "unknown"
  },
  "dataset": {
   "languages": [
    "English",
    "Chinese"
   ],
   "license": "CC BY 4.0",
   "modalities": [
    "text"
   ],
   "public_test_set": null,
   "url": "https://arxiv.org/abs/2601.16503"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna, luna-stream-a-004 (Codex coordinated)",
   "reviewed": "",
   "reviewed_by": ""
  },
  "harness": {
   "other": "MRAG-Toolkit described by the authors."
  },
  "id": "mrag",
  "measures": "The Medical Retrieval-Augmented Generation benchmark evaluates RAG systems across biomedical tasks in English and Chinese. It is designed to study how retrieval approaches, model size, and prompting affect reliability, usefulness, reasoning quality, and readability.",
  "metric": {
   "direction": "higher_is_better",
   "name": "task performance",
   "unit": "score"
  },
  "name": "MRAG",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2601.16503",
   "title": "MRAG: Benchmarking Retrieval-Augmented Generation for Bio-medicine",
   "url": "https://arxiv.org/abs/2601.16503",
   "year": 2026
  },
  "publisher": {
   "authors": [
    "Liz Li",
    "Wei Zhu"
   ],
   "org": "MRAG authors",
   "url": "https://arxiv.org/abs/2601.16503"
  },
  "released": "2026-01",
  "saturation": {
   "note": "The paper identifies retrieval and prompting effects and proposes a new biomedical benchmark.",
   "status": "open"
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "MRAG biomedical benchmark paper",
    "url": "https://arxiv.org/abs/2601.16503"
   }
  ],
  "subcategory": "biomedical retrieval-augmented generation",
  "summary": "MRAG evaluates retrieval-augmented generation for biomedical question answering in English and Chinese using Wikipedia and PubMed corpora.",
  "tags": [
   "biomedicine",
   "retrieval",
   "multilingual",
   "RAG"
  ],
  "task_format": "Biomedical retrieval-augmented question answering with long-form responses."
 }
}