OpenCompass suite of scientific sequence and text tasks from the SciReasoner paper, spanning molecules, proteins, materials, and DNA/RNA.
unassessed
| Category | domain |
|---|---|
| Subcategory | scientific translation, extraction, property, and generation tasks across molecules, proteins, materials, DNA/RNA |
| Page status | active |
| Metric | task-dependent (accuracy, ROUGE, MAD/MAE, Spearman, SMILES/IUPAC match, and others) |
| Direction | higher_is_better |
| Publisher | Open Science Lab (SciReason) |
SciReasoner here is the evaluation suite that ships with the SciReasoner foundation-model paper, not the weights. A model is asked to move between natural language and scientific objects: SMILES, formulae, proteins, DNA/RNA, and crystal or molecule properties. The paper groups work into translation, knowledge extraction, property prediction, property classification, and conditional or unconditional generation, covering up to 103 tasks. OpenCompass concatenates named slices (GUE, bio_instruction, LLM4Mat, Mol-Instructions, PEER, OPI, USPTO retrosynthesis, bulk modulus, composition, unconditional generation, and others) into `scireasoner_full_datasets` plus a mini cut of each slice.
Zero-shot GenInferencer on `{input}` unless a slice adds few-shot ICE (smol/LLM4Chem-style configs). Each slice has its own Dataset class and evaluator (accuracy, ROUGE, MAD/MAE, Spearman, SMILES match, and others). Mini configs set mini_set True. Official eval scripts live in open-sciencelab/SciReason on OpenCompass v0.4.2.
No model card in ModelSpec reports this benchmark yet.