College-level chemistry, physics and math word problems from textbooks; OpenCompass scores ten text subsets by numeric exact match after a boxed-answer parse.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | college textbook scientific problems (chemistry, physics, mathematics) with numeric answers |
| Page status | active |
| Metric | accuracy (exact match on parsed numeric answer) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 580 |
| Dataset licence | MIT |
| Publisher | University of California, Los Angeles, Caltech, and University of Washington |
SciBench asks a model to solve a collegiate chemistry, physics or mathematics exercise and return a number. Items come from named textbooks (Atkins physical chemistry, Halliday fundamentals of physics, Stewart calculus, and others). Many solutions need multi-step calculation and domain formulae, not high-school arithmetic. The paper also studies multimodal (figure) items and an error-attribution protocol over ten skills. OpenCompass implements only the text subsets: ten JSON files, four prompt styles (zero-shot, zero-shot CoT, few-shot, few-shot CoT), no figures and no Wolfram/Python tool loop.
Free-text generation. OpenCompass asks for a three-decimal number and a `\\boxed[ANSWER]` close (note the square-bracket instruction in the zero-shot template). Postprocess looks for `answer is`, then `\\boxed{...}`, then the last number in the completion. AccEvaluator does string equality with the `answer_number` field.
No model card in ModelSpec reports this benchmark yet.