NQ-CN (OpenCompass)

OpenCompass Chinese-prompt Natural Questions wrap: zero-shot short-answer generation scored by exact match on local jsonl files.

Also known as: nqcn, NaturalQuestionDatasetCN

unassessed

This page is a discovery lead. Nobody has yet assessed it against the catalogue contract, so it carries no disposition. Absence of evidence here is not evidence of staleness.

Category	knowledge
Subcategory	Chinese-prompt closed-book short-answer question answering in the Natural Questions style
Page status	unknown
Metric	exact match after OpenCompass post-processing (NQEvaluatorCN score, 0-100)
Direction	higher_is_better
Unit	%
Publisher	OpenCompass (open-compass/opencompass)

What it measures

This id is OpenCompass dataset abbr nq_cn, not English OpenCompass nq, not lm-eval nq_open, and not HELM natural_qa. The model sees a Chinese prompt of the form "问题: {question}?\n答案是：" and must emit a short answer string. The loader NaturalQuestionDatasetCN reads local ./data/nq_cn/dev.jsonl and test.jsonl. Whether those questions are translations of English Natural Questions or a new Chinese set is not stated in the config or loader. The reader evaluates the test split (train_split is set to test).

Task format

Zero-shot generation. ZeroRetriever, GenInferencer. Chinese instruction wrapper around a {question} field. pred_role BOT.

Models reporting this benchmark

No model card in ModelSpec reports this benchmark yet.

Data

This page as JSON · Edit on GitHub