An OpenCompass wrapper for MolecularIQ, which evaluates chemical reasoning over molecular structures with symbolic verification.
unassessed
| Category | domain |
|---|---|
| Subcategory | symbolically verified molecular graph reasoning |
| Page status | active |
| Metric | task-specific symbolic correctness and generation scores |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 5111 |
| Publisher | ML-JKU MolecularIQ authors |
MolecularIQ asks models to reason about molecules represented through SMILES and related symbolic forms. Its task families include counting properties, locating indexed atoms or groups, enumerating rings, and generating molecules under constraints. The benchmark focuses on questions whose answers can be checked by symbolic chemistry software rather than by a language-model judge.
OpenCompass exposes count, index, and generation configurations. Prompts require JSON inside answer tags, exact question key names, zero-based heavy-atom indices, and empty lists or zero counts when a feature is absent. Evaluators are separate for count, index, and generation tasks.
No model card in ModelSpec reports this benchmark yet.