BIG-bench self-play game: one model instance must define a target word without using forbidden related words, and a second instance must guess it.
unassessed
| Category | generation |
|---|---|
| Subcategory | constrained definition generation and comprehension, two-agent self-play word game |
| Page status | unknown |
| Metric | combined score (definition penalty + guesser reward) |
| Direction | higher_is_better |
| Dataset size | 100 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
Taboo implements the party game of the same name as a two-model self-play task. Given a target word and a list of "taboo" words closely related to it (drawn from word embeddings), one model instance must produce a definition of the target that avoids using any of the taboo words; a second model instance then reads only that definition and must guess the target word. The task jointly tests constrained, creative language generation (defining a concept under vocabulary restrictions) and language comprehension (inferring a specific concept from an indirect description), and can be run at different difficulty levels by varying how many taboo words are forbidden.
Free-response, two-phase, programmatic self-play: model A generates a definition under a word-avoidance constraint, model B (or the same model) guesses the target from that definition.
No model card in ModelSpec reports this benchmark yet.