The LAMBADA last-word task with an explicit cloze blank in the prompt, scored as log-likelihood accuracy on the withheld word.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | cloze-format last-word prediction on LAMBADA passages |
| Page status | unknown |
| Metric | accuracy (log-likelihood exact match on the last word); perplexity also reported |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 5153 |
| Dataset licence | CC BY 4.0 |
| Publisher | University of Trento (CIMeC); University of Amsterdam (dataset). EleutherAI (cloze harness configs) |
lambada_cloze is EleutherAI's cloze presentation of the LAMBADA dataset, not a new passage set. The model still sees a narrative passage whose final word is withheld, and must assign highest probability to that word. The cloze YAMLs differ from lambada_standard and lambada_openai only in the prompt: after the passage-without-last-word they insert a visible blank of the form " ____. ->" before scoring the last word. The original 2016 construction still applies: humans could guess the word from the full passage and could not from the last sentence alone.
Cloze / language modelling: passage with last word replaced by an explicit blank, scored as log-likelihood of the true last word rather than free generation or multiple choice.
No model card in ModelSpec reports this benchmark yet.