A MedHELM scenario, built from private Stanford Health Care records, that asks a model to write post-procedure patient instructions from a diagnosis, procedure and clinical notes.
unassessed
| Category | domain |
|---|---|
| Subcategory | clinical patient communication: generate personalized post-procedure discharge instructions from real clinical case data |
| Page status | active |
| Metric | LLMJuryMetric ('starr_patient_instructions_accuracy', an LLM-as-judge score from a jury of annotator models) combined with automatic summarization metrics (BERTScore-based) via HELM's summarization metric suite |
| Direction | higher_is_better |
| Unit | score |
| Dataset licence | Not public; restricted under Stanford Health Care institutional data use agreements and patient privacy protections, per the MedHELM paper |
| Publisher | Stanford Center for Research on Foundation Models (CRFM), in partnership with Stanford Health Care |
STARR Patient Instructions (displayed as "PatientInstruct" in MedHELM's taxonomy) gives a model real-world clinical case details -- a diagnosis, the planned or performed procedure, and the history-and-physical and operative notes -- and asks it to generate clear, actionable post-procedure instructions appropriate for a patient recovering from that intervention. It sits in MedHELM's "Patient Communication and Education" category and tests clinical note synthesis and patient-facing communication, not diagnostic or decision-support reasoning.
Text generation, zero-shot (all instances are assigned to the test split; no in-context training examples are drawn from the data itself). Each instance is built from a record with five fields -- Diagnosis, ActualProcedure, HistoryPhysicalNoteText, OperativeNoteText and DischargeInstructionNoteText -- filtered to records marked QC="TRUE"; the model receives the first four as input and its output is compared against the real DischargeInstructionNoteText as reference.
No model card in ModelSpec reports this benchmark yet.