CMO fill-in-the-blank

OpenCompass fill-in-the-blank set of Chinese Mathematical Olympiad problems from 2009–2022, scored with a MATH-style boxed-answer matcher.

Also known as: cmo_fib, Chinese Mathematical Olympiad FIB, CMO FIB

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	math
Subcategory	Chinese Mathematical Olympiad fill-in-the-blank problems (2009–2022)
Page status	active
Metric	MATHEvaluator v2 accuracy after math_postprocess_v2
Direction	higher_is_better
Unit	%
Publisher	OpenCompass (dataset packaging); problems from the Chinese Mathematical Olympiad

What it measures

cmo_fib is OpenCompass's generation task over fill-in-the-blank problems from the Chinese Mathematical Olympiad (CMO), contest years 2009 through 2022. The model receives the problem text as origin_prompt / question and must produce a final mathematical answer. OpenCompass does not turn the items into multiple choice. It is contest math in Chinese, closer to olympiad fill-in items than to school word problems such as GSM8K.

Task format

Zero-shot generation. Default configs (cmo_fib_gen_ace24b.py and cmo_fib_gen_2783e5.py) append a Chinese chain-of-thought request and ask for a \\boxed{} answer. cmo_fib_0shot_notcot_gen_4c6c29.py skips the "step by step" line and only asks for a boxed final answer. Dataset abbr cmo_fib; path opencompass/cmo_fib. Inferencer max_out_len is 2048 in the ace24b and 0-shot configs.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub