OpenCompass Chinese four-way Wikipedia-style MCQ set, usually reported under circular evaluation rather than single-pass accuracy.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Chinese four-way multiple-choice questions drawn from Wikipedia-style knowledge |
| Page status | unknown |
| Metric | circular accuracy (CircularEvaluator); AccEvaluator if circular is turned off |
| Direction | higher_is_better |
| Unit | % |
| Dataset licence | Apache-2.0 on OpenCompass code. Dataset licence is not stated on a public card (Hub page 404; datasets_info hf_id is empty). |
| Publisher | OpenCompass (Shanghai AI Laboratory) |
OpenCompass WikiBench is a Chinese single-choice knowledge quiz. Each item is a question with four options A-D. Prompts and few-shot exemplars are encyclopaedic (biology, places, sports, Chinese history). The default configs enable circular evaluation: the same item is asked under four option rotations and the model must get every rotation right. This page is that OpenCompass dataset, not Wikipedia-article-quality "WikiBench" work from the HCI literature.
Four-way MCQ in Chinese. Three config styles exist: generative zero-shot (wikibench_gen.py imports wikibench_gen_f96ece.py), generative chain-of-thought (wikibench_gen_0978ad.py), and few-shot perplexity (wikibench_few_shot_ppl_c23d79.py). Generation post-processes with first_option_postprocess over ABCD. CircularEvaluator is on by default (do_circular = True). Dataset class WikiBenchDataset, path opencompass/WikiBench, file single_choice_cn.jsonl.
No model card in ModelSpec reports this benchmark yet.