A 10-item BIG-bench JSON template of one-digit addition that accepts either a numeral, an English word, or both as a correct target.
unassessed
| Category | math |
|---|---|
| Subcategory | BIG-bench JSON template; addition with multiple acceptable targets |
| Page status | unknown |
| Metric | exact_str_match |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 10 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench organizers) |
simple_arithmetic_multiple_targets_json shows a one-digit addition such as "1 + 4 = " and treats the answer as correct if the model produces any of one or more listed targets. Targets mix Arabic numerals ("5") and English number words ("five"), and the list length is not the same on every item: some items have two targets, some have one, and two items list only a word ("thirteen", "four") with no numeral. The organizers wrote it to show JSON multiple-target scoring. The README notes that ROUGE is not supported with multiple targets. Tagged "example task".
Free-text completion of a one-digit addition equation; any listed target (numeral and/or English word) is accepted as correct under exact_str_match.
No model card in ModelSpec reports this benchmark yet.