About 18,000 objective and subjective questions in chemistry, physics and biology, scored on basic knowledge, application, calculation and research ability.
unassessed
| Category | domain |
|---|---|
| Subcategory | scientific knowledge and reasoning across chemistry, physics and biology |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Publisher | Shanghai Jiao Tong University (X-LANCE Lab / OpenDFM) |
SciEval tests scientific understanding across chemistry, physics and biology. Questions are organised by Bloom's-taxonomy-inspired ability levels: basic knowledge (recall of facts and definitions), knowledge application (using facts in a new context), scientific calculation (numeric problem solving) and research ability (experimental design and interpretation). Most items are multiple choice or true/false; a smaller share are open-ended judgement or fill-in questions. A "dynamic" chemistry and physics subset regenerates numeric values from templates at evaluation time so the specific numbers cannot have been memorised from a fixed test file.
Multiple choice (four options) is the dominant format via the OpenCompass integration, which reads an `input` stem with `A`-`D` options and expects a single letter answer. The original repository's `scieval-test.json` / `scieval-valid.json` also carry true/false and open fill-in-the-blank items scored by the authors' own `eval.py`, and a separate `eval_dynamic.py` regenerates and scores the templated chemistry/physics subset.
No model card in ModelSpec reports this benchmark yet.