EarthSE's harder 1,000-item Earth-science QA split; OpenCompass scores the 250 multiple-choice items, not Iron, Gold, or the other three formats.
unassessed
| Category | domain |
|---|---|
| Subcategory | hard Earth-science QA from high-impact papers (EarthSE middle tier) |
| Page status | active |
| Metric | accuracy (OpenCompass AccEvaluator on MC; paper also reports TF/FIB accuracy and FR win-rate) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1000 |
| Dataset licence | MIT |
| Publisher | Shanghai Artificial Intelligence Laboratory; Shanghai Jiao Tong University |
Earth-Silver is the professional-difficulty QA tier of EarthSE (Xu et al.). Items are English questions built from high-impact Earth-science papers, covering five spheres (atmosphere, biosphere, hydrosphere, lithosphere, cryosphere), many sub-disciplines, and 11 task types such as analysis, calculation, experiment design and code generation. Hugging Face ships four formats of 250 items each: multiple_choice, true_false, fill_in_the_blank and free_form. This id, as OpenCompass implements it, loads only split multiple_choice. The skill is hard Earth-science answering, not [climaqa](climaqa.md) climate QA and not Earth-Gold dialogue.
OpenCompass: zero-shot generation. Prompt "Q: {question}" then "output only the corresponding letter". Earth_Silver_gen.py scores AccEvaluator and mentions options A-D. LLM-judge configs mention A-E and GenericLLMEvaluator. Few-shot is NotImplemented in Earth_Silver_MCQDataset.
No model card in ModelSpec reports this benchmark yet.