HELM binary legal task: choose which of two case parentheticals more strongly supports a passage mined from US opinions.
unassessed
| Category | domain |
|---|---|
| Subcategory | binary reverse entailment over US case parentheticals |
| Page status | unknown |
| Metric | quasi_exact_match (schema); run spec also attaches exact-match metrics |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 3047 |
| Publisher | Stanford CRFM (HELM) |
LegalSupport is a comparative legal-entailment task introduced in HELM. Each item is a passage (a legal assertion) plus two parenthetical case descriptions. The model must pick the parenthetical that more forcefully supports the assertion. Labels come from Bluebook introductory signals (for example see versus see also) mined with the parentheticals from US state and federal opinions written after 1965, using the Caselaw Access Project. Neel Guha designed the scenario. English legal text. Two-way multiple choice, not [legalbench](legalbench.md).
Two references, one tagged correct. HELM default adapter: 3 in-context examples, instructions "Which statement best supports the passage?", input noun Passage, output noun Answer, method ADAPT_MULTIPLE_CHOICE_JOINT unless overridden. Binary A/B.
No model card in ModelSpec reports this benchmark yet.