MedHELM's gated wrap of MIMIC-IV-BHC: write a Brief Hospital Course from a discharge note, scored by an LLM jury plus overlap metrics.
unassessed
| Category | domain |
|---|---|
| Subcategory | discharge-note summarization into a Brief Hospital Course |
| Page status | active |
| Metric | mimic_bhc_accuracy (HELM LLM-jury average of accuracy, completeness, clarity, each 1-5) |
| Direction | higher_is_better |
| Unit | points |
| Dataset size | 270033 |
| Dataset licence | PhysioNet Credentialed Health Data License 1.5.0 (DUA 1.5.0; CITI training required) |
| Publisher | Stanford University (MIMI / CRFM MedHELM packaging) |
This id is HELM's mimic_bhc scenario, not the authors' original BLEU and BERTScore study by itself. MIMIC-IV-BHC pairs a preprocessed MIMIC-IV discharge note with the Brief Hospital Course section of that stay. HELM prompts the model to summarize the note into a BHC (zero-shot, max 1,024 tokens) and grades the text with an LLM jury on accuracy, completeness and clarity (1–5) as mimic_bhc_accuracy, while also logging summarization overlap metrics. Inputs and outputs are English clinical text.
Zero-shot generation. HELM instructions: "Summarize the clinical note into a brief hospital course." Input noun Clinical Note, output noun Brief Hospital Course. max_train_instances=0, max_tokens=1024.
No model card in ModelSpec reports this benchmark yet.