Geophysics inversion set of ~23 million Vs–dispersion pairs; OpenCompass scores two LLM configs named OpenSWI-shallow-1k and OpenSWI-deep-1k with RMSE.
unassessed
| Category | domain |
|---|---|
| Subcategory | surface-wave dispersion inversion to 1-D S-wave velocity |
| Page status | active |
| Metric | RMSE |
| Direction | lower_is_better |
| Dataset licence | CC-BY-4.0 |
| Publisher | Shanghai Jiao Tong University and Shanghai Artificial Intelligence Laboratory |
OpenSWI asks a model to recover a 1-D S-wave velocity (Vs) profile from a fundamental-mode surface-wave dispersion curve. The original dataset is a geophysics benchmark for deep inversion networks, not a reading quiz. OpenCompass wraps two subsets as chat generation: a system prompt names a geophysical inversion expert, the user message is the stored prompt, and the model must emit a Python list of Vs values. Shallow pairs come from OpenFWI 2-D geology (README table 0.1–10 s; Hugging Face and ESSD text 0.2–10 s, depths to about 2.8 km). Deep pairs come from 14 global and regional 3-D models. OpenSWI-real holds observed curves from Long Beach and the China Seismological Reference Model; OpenCompass does not load that split.
OpenCompass: zero-shot GenInferencer with ZeroRetriever. Configs openswi_gen and openswi_rawprompt_gen differ only in PromptTemplate versus RawPromptTemplate. Both ask for a Python list. OpenSWIMSEEvaluator parses the last bracketed list, pads or trims to the gold length, and reports RMSE plus a validity rate. The paper's own protocol trains a transformer on millions of synthetic pairs and tests on OpenSWI-real, which is a different setup.
No model card in ModelSpec reports this benchmark yet.