HELM generation task: read an English COVID-19 patient question and write the doctor's reply, scored with overlap metrics.
unassessed
| Category | domain |
|---|---|
| Subcategory | HELM English patient-to-doctor response generation on COVID-19 consultations |
| Page status | unknown |
| Metric | open-ended overlap (exact_match, quasi_exact_match, f1_score, rouge_l, bleu_1, bleu_4) |
| Direction | higher_is_better |
| Dataset size | 603 |
| Publisher | UCSD AI4H (dataset); Stanford CRFM (HELM scenario) |
covid_dialog is HELM's wrap of the English CovidDialog consultations, not a new item set. The model reads a patient's COVID-19 or pneumonia concern and must write the doctor's reply. HELM strips a leading "patient: " from the source line and prompts with Patient / Doctor. The original English collection is described as 603 consultations with id, URL, condition description, and dialogue. This is English doctor-response generation, not the Chinese Haodf.com dump and not HELM med_dialog.
Instruction "Generate a response given a patient's questions and concerns." then Patient: … / Doctor: … . Default five in-context examples, temperature 0, max_tokens 128. Run spec name covid_dialog; group COVIDDialog.
No model card in ModelSpec reports this benchmark yet.