OpenCompass suite of Chinese professional-licence and kaoyan exam items in six formats, scored zero-shot with regex extraction; item count is not published.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Chinese professional-licence and postgraduate-entrance exam items in six formats |
| Page status | unknown |
| Metric | KaoshiEvaluator score (exact match after regex extraction; partial credit on multi-choice) |
| Direction | higher_is_better |
| Unit | % |
| Publisher | OpenCompass / Shanghai AI Laboratory (harness packaging); original exam authors not named in the loader |
kaoshi, in this id, is OpenCompass's KaoshiDataset: Chinese exam questions drawn from professional licensing tracks and from China's postgraduate entrance examination (kaoyan), not from the national college entrance exam that [gaokaobench](gaokaobench.md) uses. Single-choice splits cover fire safety, surveying, safety engineering, architecture, teacher qualification, securities, accounting, civil service, project management (高项), banking, funds, and kaoyan economics, politics, English, mathematics, clinical medicine and comprehensive management. Other formats appear only on some of those papers. The skill is exam-style answering in Chinese (with English on the kaoyan English paper), not a translated MMLU subject.
Zero-shot generation (OpenCompass ZeroRetriever + GenInferencer, max_out_len 1024). Chinese prompts ask the model to put reasoning between 【解析】 and <eoe> and the final answer between 【答案】 and <eoa>, with format examples that differ by question type. KaoshiEvaluator then regex-extracts letters or short strings from the completion.
No model card in ModelSpec reports this benchmark yet.