OpenCompass LLM wrap of MP-20: emit lattice and atomic sites as JSON, scored by StructureMatcher match rate and RMS.
unassessed
| Category | domain |
|---|---|
| Subcategory | OpenCompass LLM crystal-structure reconstruction on MP-20, scored with StructureMatcher |
| Page status | unknown |
| Metric | match_rate |
| Direction | higher_is_better |
| Unit | % |
| Publisher | OpenCompass (LLM wrap); MP-20 curated by Xie et al. from the Materials Project |
OpenCompass mp20 asks a language model to write a crystal as JSON: lattice parameters and atomic-site coordinates. The evaluator parses that JSON, builds a pymatgen Structure, and matches it to a ground-truth structure with the same element list. The underlying crystals are MP-20, the Materials Project slice with at most 20 atoms per cell, curated for CDVAE. This is structure reconstruction as text, not [matbench](matbench.md) property prediction, and not unconstrained de novo generation without a reference cell.
Zero-shot GenInferencer with RawPromptTemplate. Messages are a system role from {system_prompt} and a user role from {user_input}. ZeroRetriever. abbr mp20. Dataset class MP20Dataset reads a JSONL via get_data_path('opencompass/mp20'). Evaluator MP20Evaluator, pred_role BOT. JSON may be wrapped in <material> tags, a markdown fence, or a trailing large JSON object.
No model card in ModelSpec reports this benchmark yet.