{
 "body": "\n## What it measures\n\nThis benchmark measures paragraph-level simplification of medical text. A technical clinical abstract is provided, and the model must produce a plain-language summary that remains faithful to the evidence and conclusions.\n\nThe corpus covers clinical topics and stores aligned abstract and plain-language-summary files. The source describes `pls` as \u201cplain-language summary.\u201d\n\n## How it is scored\n\nHELM creates one reference summary for each source-target pair. The scenario does not declare a main metric or evaluator. Any reported ROUGE, BLEU, readability, or semantic score must therefore include its own implementation and normalization details.\n\n## Dataset and licence\n\nThe scenario downloads `train.source`, `val.source`, and `test.source`, paired with target files from the official repository\u2019s `data/data-1024` directory, including the test split, so the reference summaries are publicly visible rather than held out. It does not state item counts. The repository\u2019s root `LICENSE.md` is the Creative Commons Attribution 4.0 International licence (CC BY 4.0); the examples include DOI-linked clinical-review abstracts, but the source does not establish rights for every underlying text beyond that repository licence.\n\n## Who publishes it\n\nDevaraj, Marshall, Wallace, and Li introduced the corpus in the 2021 NAACL paper \u201cParagraph-level Simplification of Medical Texts.\u201d The authors maintain the linked repository. HELM provides the runnable scenario; no current leaderboard was established.\n\n## Lineage\n\nThis is a standalone medical simplification corpus. The sources do not identify a predecessor, successor, or formal variant.\n\n## Saturation and contamination\n\nSaturation is unknown. The repository makes data accessible, creating some exposure risk, but no contamination study was located. Medical simplification quality can also depend on factual preservation and reading level, which a single automatic score may miss.\n\n## How to run it\n\nUse HELM\u2019s `med_paragraph_simplification` scenario. It downloads aligned source and target files for train, validation, and test, then emits abstract-to-summary generation instances. Record the evaluator and any readability constraints because the scenario itself does not choose a metric.\n\n## Reading the numbers\n\nA good result suggests that a model can shorten technical medical prose for a general reader. It does not prove that the output is clinically safe, factually complete, or understandable to a specific patient population. Inspect omissions, dosage claims, uncertainty, and readability alongside aggregate scores.\n\nThe source and target are aligned line by line, which makes evaluation sensitive to segmentation and whitespace handling. Reproducible runs should preserve the repository files and document preprocessing before scoring.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "med_paragraph_simplification",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "MedParaSimplification"
  ],
  "category": "generation",
  "contamination": {
   "note": "The corpus is public through the repository; model exposure was not established.",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "English"
   ],
   "license": "CC-BY-4.0",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": null,
   "size_note": "The source code downloads data-1024 train, val, and test source/target files but does not state counts.",
   "splits": "train",
   "test": null,
   "url": "https://github.com/AshOlogn/Paragraph-level-Simplification-of-Medical-Texts",
   "validation": null
  },
  "freshness": {
   "luna-batch-057 (Codex coordinated)": null,
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review, luna-batch-057"
  },
  "harness": {
   "bigbench": "",
   "helm": "med_paragraph_simplification",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "",
   "other": ""
  },
  "id": "med_paragraph_simplification",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "The task asks a model to rewrite a technical abstract as a plain-language summary while preserving its important clinical content. The corpus pairs abstracts with human-written or author-provided lay summaries.",
  "metric": {
   "baseline_note": "HELM does not specify a main metric in this scenario.",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": null,
   "name": "",
   "random_baseline": null,
   "unit": ""
  },
  "name": "Paragraph-level Simplification of Medical Texts",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2104.05767",
   "title": "Paragraph-level Simplification of Medical Texts",
   "url": "https://arxiv.org/abs/2104.05767",
   "year": 2021
  },
  "publisher": {
   "authors": [
    "Ashwin Devaraj",
    "Iain Marshall",
    "Byron Wallace",
    "Junyi Jessy Li"
   ],
   "org": "Association for Computational Linguistics / Ashwin Devaraj and colleagues",
   "url": "https://aclanthology.org/2021.naacl-main.395/"
  },
  "released": "2021",
  "repo_url": "https://github.com/AshOlogn/Paragraph-level-Simplification-of-Medical-Texts",
  "saturation": {
   "as_of": "",
   "note": "No current authoritative leaderboard was established.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "HELM medical paragraph simplification scenario",
    "url": "https://raw.githubusercontent.com/stanford-crfm/helm/main/src/helm/benchmark/scenarios/med_paragraph_simplification_scenario.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "NAACL paper",
    "url": "https://aclanthology.org/2021.naacl-main.395/"
   },
   {
    "accessed": "2026-09-08",
    "title": "Official repository",
    "url": "https://github.com/AshOlogn/Paragraph-level-Simplification-of-Medical-Texts"
   },
   {
    "accessed": "2026-09-08",
    "title": "Repository LICENSE.md (CC BY 4.0)",
    "url": "https://raw.githubusercontent.com/AshOlogn/Paragraph-level-Simplification-of-Medical-Texts/main/LICENSE.md"
   }
  ],
  "status": "active",
  "subcategory": "medical text simplification",
  "summary": "This benchmark maps technical medical abstracts to lay-language summaries across clinical topics.",
  "tags": [
   "medical",
   "simplification",
   "summarization"
  ],
  "task_format": "Medical abstract input and plain-language summary target."
 }
}