English legal NLU suite of seven public datasets; HELM scores each subset as generation with classification_macro_f1 on test.
unassessed
| Category | domain |
|---|---|
| Subcategory | seven-task English legal NLU suite (ECtHR A/B, SCOTUS, EUR-LEX, LEDGAR, UNFAIR-ToS, CaseHOLD) |
| Page status | active |
| Metric | classification_macro_f1 (HELM schema); papers also report micro-F1 |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 23607 |
| Dataset licence | CC-BY-4.0 |
| Publisher | University of Copenhagen and co-authors (LexGLUE); Stanford CRFM (HELM scenario) |
LexGLUE packs seven English legal datasets behind one evaluation recipe. ECtHR A predicts violated Convention articles from facts. ECtHR B predicts articles the court considered. SCOTUS maps an opinion to a Supreme Court Database issue area. EUR-LEX assigns EuroVoc labels to an EU act. LEDGAR classifies an SEC contract provision into one of 100 topics. UNFAIR-ToS tags unfair term types in a consumer ToS sentence. CaseHOLD is five-way holding identification. HELM prompts these as generation, not encoder fine-tuning.
HELM run spec lex_glue:subset=<ecthr_a|ecthr_b|scotus|eurlex|ledgar|unfair_tos|case_hold> or subset=all. Scenario loads Hugging Face config lex_glue. Generation adapter, input noun Passage, output noun Answer. MLTC tasks use comma separated labels; CaseHOLD is QA with numbered endings.
No model card in ModelSpec reports this benchmark yet.