HELM Enterprise binary task: given English ECHR facts, answer Yes or No whether any Convention article was violated, using Chalkidis et al. 2019 data.
unassessed
| Category | domain |
|---|---|
| Subcategory | binary ECHR article-violation classification of case facts |
| Page status | unknown |
| Metric | classification_weighted_f1 (schema); also exact match |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 11478 |
| Publisher | Athens University of Economics and Business; University of Sheffield (dataset); Stanford CRFM (HELM Enterprise scenario) |
echr_judgment_classification is HELM's wrap of the binary violation task in Chalkidis, Androutsopoulos and Aletras (ACL 2019). The model reads concatenated fact paragraphs from a European Court of Human Rights case and must answer Yes if any article or protocol was violated and No otherwise. HELM does not score the paper's multi-label article task or importance regression. English legal facts. The enterprise run spec calls this a different implementation of lex_glue_fixed:subset=ecthr_a; LexGLUE ECtHR-A is multi-label article prediction, so that comment is not an alias for this id.
Generation adapter with instructions "Is the following case a violation of human rights?" plus two trivial Yes/No examples in the instruction block. Input noun Case, output noun Answer, max_tokens=1, default max_train_instances=5. HELM keeps documents of at most 600 words (filter_max_length=600; scenario code splits on non-word characters). Labels Yes/No from non-empty VIOLATED_ARTICLES.
No model card in ModelSpec reports this benchmark yet.