Legal Contract Summarization (HELM)

HELM generation task: rewrite a short unilateral-contract snippet in plain English and score ROUGE-L against community summaries.

Also known as: Plain English Summarization of Contracts

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	domain
Subcategory	HELM Enterprise plain-English summarization of TOS and license snippets
Page status	unknown
Metric	rouge_l
Direction	higher_is_better
Dataset size	446
Publisher	University of Texas at Austin (dataset); Stanford CRFM (HELM Enterprise scenario)

What it measures

legal_contract_summarization is HELM's wrap of Manor and Li (NLLP 2019). The model reads a cleaned snippet from a terms-of-service or software license and must write a short plain-English summary. Gold text comes from TL;DRLegal and TOS;DR community pages, not from a court. English. This is not [legal_summarization](legal_summarization.md) (BillSum / MultiLexSum / EurLexSum).

Task format

HELM @run_spec_function legal_contract_summarization in enterprise_run_specs.py. Instructions "Summarize the legal document in plain English." Input noun Document, output noun Summary, max_tokens 100, stop at blank line. Scenario.name matches the run spec.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub