GovRepcrs (OpenCompass / GovReport CRS)

OpenCompass English summarization of Congressional Research Service reports from GovReport, scored with BLEU rather than ROUGE.

Also known as: GovRepcrs, govrepcrs_gen, GovReport CRS

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	long-context
Subcategory	OpenCompass BLEU summarization of Congressional Research Service reports
Page status	unknown
Metric	BLEU
Direction	higher_is_better
Publisher	University of Michigan and University of Illinois Urbana-Champaign (GovReport); OpenCompass (harness wrap)

What it measures

govrepcrs is OpenCompass's loader for the Congressional Research Service (CRS) half of GovReport (Huang, Cao, Parulian, Ji, and Wang, NAACL 2021). The model reads a long English CRS report (title plus nested section paragraphs) and writes an English summary. It does not load GAO reports, which are the other half of GovReport. The skill is long-document summarization of US policy reports, not short news summarization.

Task format

Zero-shot generation. Prompt: "Please summarize the following English report in English:" then the content. GenInferencer max_out_len 500, max_seq_len 8192, batch_size 4. ZeroRetriever. Reader uses test for both train_split and test_split. Dataset abbr GovRepcrs. Path ./data/govrep/.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub