MedHELM's gated wrap of the BioNLP 2024 DischargeMe shared task: write Brief Hospital Course and Discharge Instructions from MIMIC-IV notes and radiology text.
unassessed
| Category | domain |
|---|---|
| Subcategory | discharge-instruction and brief-hospital-course generation from MIMIC-IV notes |
| Page status | active |
| Metric | dischargeme_accuracy (HELM LLM-jury mean of accuracy, completeness, clarity, each 1-5) |
| Direction | higher_is_better |
| Unit | points |
| Dataset size | 14702 |
| Dataset licence | PhysioNet Credentialed Health Data License 1.5.0 (DUA 1.5.0; CITI training required) |
| Publisher | Stanford AIMI (shared task); Stanford CRFM (MedHELM scenario) |
This id is HELM's dischargeme scenario, not the Codabench shared-task score by itself. Each item is a MIMIC-IV emergency admission. HELM strips the gold target section from the discharge note, pairs the remainder with one radiology report, and asks the model to write either the Brief Hospital Course or the Discharge Instructions. English clinical text. The intended skill is clinically accurate generation of those two discharge-summary sections, not full-note drafting and not the separate MIMIC-BHC corpus.
Zero-shot generation (max_train_instances=0). HELM instructions: given discharge text, radiology text, and a named target document, return that document. max_tokens=300 by default; some gated run entries raise num_output_tokens to 4000. Two HELM instances per remaining admission (one per target section).
No model card in ModelSpec reports this benchmark yet.