{
 "body": "\n## What it measures\n\nThe molecule-oriented Mol-Instructions tasks test whether a model can follow chemistry instructions involving small molecules. The six tasks cover describing a molecule, designing a molecule from a description, predicting products, predicting reagents, retrosynthesis, and predicting a molecular property. Inputs and outputs can use SELFIES or natural language.\n\nThis catalogue ID documents the OpenCompass chemistry configuration. It is a task wrapper around one component of the wider Mol-Instructions release, which also contains protein-oriented and biomolecular-text instructions.\n\n## How it is scored\n\nOpenCompass runs the six tasks zero-shot. Molecular outputs use a SELFIES-aware evaluator that decodes the strings and compares Morgan fingerprints with Tanimoto similarity. Property prediction uses mean absolute error. Molecular descriptions use METEOR. These metrics have different directions and should be reported per task.\n\nThe configuration requires task-specific output formatting: tagged SELFIES for molecular outputs, a boxed number for property prediction, and natural language for descriptions. A malformed output can therefore receive a low score even when its underlying chemistry is plausible.\n\n## Dataset and licence\n\nThe official README reports 148.4K molecule-oriented instructions across six tasks. It also reports separate protein-oriented and biomolecular-text components, which are outside this page. The README states that the dataset is CC BY 4.0 and restricted to research use. Exact counts for each OpenCompass test file were not established, so no per-task split count is asserted.\n\n## Who publishes it\n\nYin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen introduced Mol-Instructions. Zhejiang University NLP maintains the repository and Hub release. OpenCompass supplies the runnable `MolInstructions_chem` configuration.\n\n## Lineage\n\nMol-Instructions is a standalone biomolecular instruction dataset and evaluation release. The molecule-oriented tasks are distinct from [molculariq](molculariq.md), whose questions are symbolically verified molecular-graph tasks. Later chemistry instruction datasets exist, but no direct successor is established here.\n\n## Saturation and contamination\n\nPublished results compare instruction-tuned models on several molecule-oriented tasks, but no current public leaderboard was established. The test files and many source datasets are public, so contamination risk is medium. The release does not report a measured leakage study.\n\n## How to run it\n\nUse OpenCompass\u2019s `MolInstructions_chem/mol_instructions_chem_gen.py`. It loads the `MolInstructionsDataset` task files, applies zero-shot chemistry prompts, and evaluates the six task names: reagent prediction, molecule design, forward reaction prediction, retrosynthesis, property prediction, and molecular description generation. Match the SMILES versus SELFIES configuration and evaluator before comparing results.\n\n## Reading the numbers\n\nA high molecular similarity score means the generated molecule is structurally close under the evaluator\u2019s fingerprint representation. A low MAE means better property prediction, while higher METEOR indicates closer text descriptions. None of these scores alone establishes laboratory validity, reaction yield, or safe use of a generated molecule.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "molinstructions_chem",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "MolInstructions_chem",
   "Mol-Instructions",
   "OpenCompass MolInstructions chem"
  ],
  "category": "domain",
  "contamination": {
   "note": "The instruction files and source datasets are public. The release is intended for research and does not provide a measured contamination study.",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "en"
   ],
   "license": "CC BY 4.0",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 148400,
   "size_note": "The official Mol-Instructions README describes 148.4K molecule-oriented instructions across six tasks. OpenCompass loads task JSONL files from the molecule-oriented release.",
   "splits": "Molecule-oriented task files with train/test releases; exact OpenCompass per-task counts were not established.",
   "url": "https://huggingface.co/datasets/zjunlp/Mol-Instructions"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna, luna-batch-059 (Codex coordinated)",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review, luna-batch-059"
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "MolInstructions_chem",
   "other": ""
  },
  "id": "molinstructions_chem",
  "last_updated": "2024-10",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "This benchmark evaluates instruction following for small-molecule chemistry. The molecule-oriented component contains molecule description generation, description-guided molecule design, forward reaction prediction, retrosynthesis, reagent prediction, and property prediction. Inputs use SELFIES in the OpenCompass configuration and outputs are molecular strings, numbers, or natural-language descriptions.\n",
  "metric": {
   "baseline_note": "Property prediction uses lower-is-better MAE; molecular and text generation use higher-is-better scores.",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": null,
   "name": "Tanimoto similarity, mean absolute error, or METEOR by task",
   "random_baseline": null,
   "unit": "task-specific"
  },
  "name": "Mol-Instructions molecule-oriented tasks",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2306.08018",
   "title": "Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models",
   "url": "https://arxiv.org/abs/2306.08018",
   "year": 2023
  },
  "publisher": {
   "authors": [
    "Yin Fang",
    "Xiaozhuan Liang",
    "Ningyu Zhang",
    "Kangwei Liu",
    "Rui Huang",
    "Zhuo Chen",
    "Xiaohui Fan",
    "Huajun Chen"
   ],
   "org": "Zhejiang University NLP",
   "url": "https://github.com/zjunlp/Mol-Instructions"
  },
  "released": "2023-06",
  "repo_url": "https://github.com/zjunlp/Mol-Instructions",
  "saturation": {
   "as_of": "",
   "note": "The source reports task results for instruction-tuned models, but no current comparable leaderboard was established.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "Mol-Instructions paper",
    "url": "https://arxiv.org/abs/2306.08018"
   },
   {
    "accessed": "2026-09-08",
    "title": "Official Mol-Instructions README",
    "url": "https://raw.githubusercontent.com/zjunlp/Mol-Instructions/main/README.md"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass MolInstructions chemistry configuration",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/MolInstructions_chem/mol_instructions_chem_gen.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass MolInstructions dataset and evaluators",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/datasets/mol_instructions_chem.py"
   }
  ],
  "status": "active",
  "subcategory": "molecule-oriented biomolecular instruction evaluation",
  "summary": "An OpenCompass wrapper for six molecule-oriented Mol-Instructions tasks covering molecular descriptions, design, reactions, properties, and retrosynthesis.",
  "tags": [
   "chemistry",
   "biomolecular",
   "instruction-following",
   "molecular-generation",
   "opencompass"
  ],
  "task_format": "Zero-shot generation. OpenCompass formats each task with a chemistry system prompt and asks for SELFIES tags for molecular outputs, a boxed number for property prediction, or natural language for descriptions. It evaluates molecular outputs with Morgan-fingerprint Tanimoto similarity, numbers with MAE, and descriptions with METEOR.\n"
 }
}