BIG-bench RoFT task: a 10-sentence English passage starts human-written; pick the first machine-generated sentence.
unassessed
| Category | generation |
|---|---|
| Subcategory | BIG-bench multiple-choice: find the sentence where a passage switches from human text to generated text |
| Page status | unknown |
| Metric | multiple_choice_grade |
| Direction | higher_is_better |
| Dataset size | 15088 |
| Dataset licence | Apache-2.0 |
| Publisher | University of Pennsylvania (via the BIG-bench collaboration) |
real_or_fake_text (RoFT) shows ten English sentences. The first is always human. At a random later sentence the rest become model-generated, or, in the easy subtask, sentences copied from another human article in the same genre. The model must name the first machine-generated sentence, without being told which generator was used. This is boundary detection in a hybrid passage, not a binary real/fake label. Subtasks: easy, gpt2, gpt2_xl. Not in [bbh](bbh.md).
JSON multiple choice over the passage's own sentences. Preferred metric multiple_choice_grade. Task prefix matches the human RoFT instructions. Dummy-model header: 15,088 multiple-choice queries. Parent task.json is a stub; examples live in easy/, gpt2/, and gpt2_xl/. Canary GUID embedded.
No model card in ModelSpec reports this benchmark yet.