An 85-item BIG-bench free-response task: given a Codenames-style clue and a word list, emit the associated words in alphabetical order.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | BIG-bench Codenames-style free-response word association (85 items) |
| Page status | unknown |
| Metric | bleu |
| Direction | higher_is_better |
| Dataset size | 85 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
codenames gives one English clue word and a board of candidate words, then asks the model to name the words that best match the clue. Isaac Noble, Lucy Noble, Emma Lam, and Lucas Lam wrote the clues and boards for BIG-bench; they are not dumps of the board game. The probe is analogical word association, not playing a full two-team Codenames match.
Free-text list. Preferred metric bleu (rouge is also listed). Inputs ask for 1, 2, 3, or 4 associated words and require alphabetical order. No multiple-choice options. Canary GUID embedded.
No model card in ModelSpec reports this benchmark yet.