Nine chemistry generation subtasks (45,000 instructions) that ask for SMILES molecules satisfying atom, edit, or property constraints, scored with RDKit success and weighted success.
unassessed
| Category | domain |
|---|---|
| Subcategory | open-domain natural-language molecule generation (MolCustom, MolEdit, MolOpt) |
| Page status | active |
| Metric | weighted success rate (WSR); success rate (SR) and validity also reported |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 45000 |
| Dataset licence | Apache-2.0 |
| Publisher | Authors of Speak-to-Structure (KDD 2026); Hugging Face dataset phenixace |
S2-TOMG-Bench (Speak-to-Structure; also called S²-Bench) asks a model to invent a molecule from an English instruction and return it as SMILES, usually as JSON `{"molecule": "..."}`. Three families probe different skills: MolCustom (build from atom, bond, or functional-group counts), MolEdit (add, delete, or substitute a group on a source molecule), and MolOpt (raise or lower LogP, molar refractivity, or QED). The point is one-to-many generation, not retrieving a single stored SMILES. OpenCompass wraps the nine CSV configs as separate generation tasks plus a 10% mini pack.
Zero-shot chat: a chemist-assistant system prompt plus the instruction. The expected final span is a JSON object with key molecule. OpenCompass uses RawPromptTemplate and GenInferencer. Scoring is automatic with RDKit, not an LLM judge.
No model card in ModelSpec reports this benchmark yet.