MedHELM private NoteExtract task: rewrite a community health worker care-plan note into a fixed clinical template, scored by an LLM jury.
unassessed
| Category | domain |
|---|---|
| Subcategory | structured extraction from community-health-worker care-plan notes |
| Page status | active |
| Metric | chw_care_plan_accuracy (HELM LLM-jury average of accuracy, structure, clarity, each 1-5) |
| Direction | higher_is_better |
| Unit | points |
| Publisher | Stanford CRFM / MedHELM |
This id is HELM's chw_care_plan scenario, shown on the MedHELM leaderboard as NoteExtract. It is not MTSamples, MIMIC-BHC, or a Stanford Health Care shc_* task. Each item is an English free-form care-plan note in the CSV column "MO Note". The prompt asks the model to extract chief complaint and history-of-present-illness fields (onset, provoking/palliating factors, quality, region/radiation, severity, timing, related symptoms) and to write "Not mentioned" rather than infer. The paper describes the same work as restructuring community health worker care plans into a specified format without a gold-standard response.
Zero-shot generation. HELM adapter instructions: "Follow the instructions provided regarding conversion of a patient note into a specified format." Empty input and output nouns. max_train_instances=0, max_tokens=768. Official private run entries pass data_path=/share/pi/nigam/datasets/CHW_Dataset.csv.
No model card in ModelSpec reports this benchmark yet.