Rephrase (BIG-bench)

78 English keyword-transformation items: rewrite a sentence so it keeps the meaning and contains a fixed keyword.

Also known as: Keyword Sentence Transformation

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	generation
Subcategory	BIG-bench keyword-constrained sentence paraphrase (C2/C1-style)
Page status	unknown
Metric	log_likelihood
Direction	higher_is_better
Dataset size	78
Dataset licence	Apache-2.0
Publisher	Google (BIG-bench collaboration)

What it measures

rephrase shows a lead-in sentence and a keyword. The model must rewrite the sentence so the meaning stays close and the keyword appears unchanged. Author Marie Tolkiehn. The README frames it as Cambridge C1 Advanced / C2 Proficiency Use of English Part 4. Not a general paraphrase corpus.

Task format

Free-response JSON. Preferred metric log_likelihood (anchors low_score -10, high_score 0 in BIG-bench score files). Also reports BLEU, ROUGE, and exact_str_match. Dummy-model header: 78 free-text queries. Canary GUID embedded.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub