An OpenCompass Chinese exam suite of 2022-2023 Gaokao and 2022 Zhongkao multiple-choice items, scored from a chain-of-thought answer letter; the public dataset class is missing.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | Chinese Gaokao and Zhongkao multiple-choice exams (2022-2023) |
| Page status | unknown |
| Metric | not established in public code (PJExamEvaluator is referenced but not published) |
| Direction | higher_is_better |
PJExam, as implemented in OpenCompass configs, is a Chinese multiple-choice examination evaluation. The seven named splits are Gaokao papers from 2022 and 2023 (full paper and math-only cuts, including two 2023 versions) and a 2022 Zhongkao paper. The model is prompted in Chinese to write a chain-of-thought between 【解析】 and <eoe>, then a single letter from A-D between 【答案】 and <eoa>. What "PJ" expands to is not stated in the public files. Item content, subject mix and official scoring rules were not inspectable because the loader and the local data dump are not in the public repository.
Zero-shot Chinese generation. OpenCompass uses GenInferencer with max_out_len 1024 and ZeroRetriever. The published hint string substitutes a </major> token into "请你做一道</major>选择题" and requires the rigid 【解析】/【答案】 markup. Evaluator class name: PJExamEvaluator; gold column std_ans; extra eval payload in ds_column eval_infos.
No model card in ModelSpec reports this benchmark yet.