lm-eval wrap of MTS-Dialog: write an English clinical-note section from a short doctor-patient dialogue, scored with overlap metrics.
unassessed
| Category | domain |
|---|---|
| Subcategory | lm-eval clinical-note section generation from short doctor-patient dialogue |
| Page status | active |
| Metric | BLEU, ROUGE-1/2/L, BERTScore, BLEURT (nanmean); perplexity on the companion task |
| Direction | higher_is_better |
| Dataset size | 1301 |
| Dataset licence | CC-BY-4.0 on abachaa/MTS-Dialog; MIT on Hugging Face har1/MTS_Dialogue-Clinical_Note |
| Publisher | Microsoft (original dataset); EleutherAI lm-evaluation-harness wrap; Hugging Face mirror by har1 |
mts_dialog is EleutherAI lm-evaluation-harness's generation wrap of MTS-Dialog, a 2023 collection of short English doctor-patient dialogues paired with clinical-note sections. The model reads the dialogue and must write the corresponding section text. The original notes use twenty normalised headers (chief complaint, history of present illness, and so on). The Hugging Face mirror that lm-eval loads reformats the note as Symptoms / Diagnosis / History of Patient / Plan of Action. This is section-level note writing, not full-visit notes as in [aci_bench](aci_bench.md).
generate_until, stop at a blank line. doc_to_text is the dialogue; doc_to_target is section_text. YAML instruction: extract a note that summarises the dialog. training_split, validation_split, and test_split are all `train` on har1/MTS_Dialogue-Clinical_Note. Companion task mts_dialog_perplexity uses loglikelihood_rolling.
No model card in ModelSpec reports this benchmark yet.