A 323-item BIG-bench task that classifies how two short event sentences share objects, relations, and structure, using Gentner-style similarity labels.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | BIG-bench seven-way analogical similarity of sentence episodes (323 items) |
| Page status | unknown |
| Metric | multiple_choice_grade |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 323 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
analogical_similarity gives two short English "episodes" (nested cause/effect event sentences) and asks which of seven similarity classes best describes the pair: literal similarity, analogy, cross mapping, surface similarity, false analogy, only-objects similarity, or no similarity. Authors Cameron Diao and Denis Kleyko built the items from Plate's holographic-reduced- representation scheme so that object attributes, first-order relations, and higher-order structure can be present or absent independently. The skill is structured analogy, not SAT word analogies.
Seven-option multiple choice, preferred metric multiple_choice_grade. task_prefix explains the rating scheme and gives a literal-similarity example. append_choices_to_input is false. Keywords include many-shot. Canary GUID embedded.
No model card in ModelSpec reports this benchmark yet.