A 39-item BIG-bench task that asks which listed order of real-world events or entities is chronological; internally titled Sequential Order.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | BIG-bench chronological and sequential-order multiple-choice (39 items) |
| Page status | unknown |
| Metric | multiple_choice_grade |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 39 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
logical_sequence (README title "Sequential Order") asks a model to pick the correct chronological or otherwise sequential order of a short list -- sock then shoe then lace, funding rounds, US presidents, non-consecutive weekdays. Authors Debajyoti Datta and Trishala Neeraj designed the items so the order is often non-adjacent, to reduce solving by repeating common web n-grams such as "Monday Tuesday Wednesday." They describe it as mixing world knowledge, encyclopedic order and magnitude, not as a formal logic puzzle.
Multiple-choice with a single correct ordering. task.json has 38 four-option items and one three-option item. Preferred metric is `multiple_choice_grade`. Canary GUID embedded. The README "Data source" section is empty; no provenance beyond the task files was stated.
No model card in ModelSpec reports this benchmark yet.