Private MedHELM task: decide whether a patient-portal message contains confidential or privacy-leaking information, answering A for yes or B for no.
unassessed
| Category | domain |
|---|---|
| Subcategory | confidential content in patient portal messages |
| Page status | active |
| Metric | exact_match |
| Direction | higher_is_better |
| Unit | 0-1 scale |
| Dataset size | 300 |
| Publisher | Stanford University Department of Pediatrics, Stanford Health Care, and Stanford CRFM (MedHELM) |
PrivacyDetection tests whether a model can read an English patient-portal message and say whether it contains confidential or privacy-leaking information that should be protected. HELM metadata describes messages from patients or caregivers. The census id is shc_privacy; the runnable HELM scenario is shc_privacy_med.
Joint multiple-choice generation. HELM instructs the model to review clinical messages for confidential information and answer A for yes or B for no.
No model card in ModelSpec reports this benchmark yet.