{
 "body": "\nPart of the [FewCLUE](fewclue.md) family.\n\n## What it measures\n\nCHID (Chinese IDiom cloze test) gives the model a passage with one blank marked #idiom# and seven\ncandidate idioms, several of them near-synonyms of the correct answer chosen specifically to block\nshallow pattern matching; the model must pick the one that actually fits the context. Passages are\ndrawn from news, novels and essays, sampled from a larger idiom-cloze corpus covering 3,848 idioms\n(Zheng et al. 2019, arXiv:1906.01265). CHID is one of six FewCLUE tasks resampled from the CLUE\nbenchmark rather than newly built, and because a blank slot with candidates is already cloze-shaped,\nthe paper notes that training on CHID with PET comes close to zero-shot prompting in practice.\n\n## Reading the numbers\n\nThe labelled public test set holds 2,002 items, scored by accuracy against a 14.3% seven-way random\nbaseline. Human evaluators scored 87.1%; the paper's few-shot methods clustered around 57-61% (PET\nand LM-BFF both reached 61.2%), well below human performance but far above chance, while plain\nfine-tuning badly underperformed at under 15% -- direct fine-tuning could not make good use of 42\ntraining examples on a task this format-sensitive. A score close to 15% suggests a fine-tuning-style\napproach rather than a cloze-native one. See the [FewCLUE](fewclue.md) family page for the shared\nfew-shot protocol and contamination notes that apply here too.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "fewclue_chid",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "category": "reasoning",
  "dataset": {
   "languages": [
    "zh"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 2002,
   "size_note": "2,002 labelled public test items (test_public.json, used for scoring), plus 2,000 in the private test set (original leaderboard only), 42 train and 42 dev examples per split (5 parallel splits, train_0..train_4 / dev_0..dev_4, plus a merged train_few_all/dev_few_all), and 7,585 unlabelled items not used for scoring. Figures from the FewCLUE paper's Table 1, matching the GitHub README. The underlying idiom-cloze format and its 3,848-idiom candidate pool originate in ChID (Zheng et al. 2019, arXiv:1906.01265), sampled here from news, novels and essays.\n",
   "splits": "train_0..train_4 (42 each) + train_few_all; dev_0..dev_4 (42 each) + dev_few_all; test_public (2,002, labelled); test (2,000, private); unlabeled (7,585)",
   "url": "https://github.com/CLUEbenchmark/FewCLUE/tree/main/datasets/chid"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "sonnet-5 agent, batch 5, slice B"
  },
  "harness": {
   "opencompass": "FewCLUE_chid"
  },
  "id": "fewclue_chid",
  "leaderboard_url": "https://www.cluebenchmarks.com/fewclue.html",
  "lineage": {
   "family": "fewclue",
   "predecessor": ""
  },
  "measures": "A Chinese passage with one blank marked #idiom# and seven candidate idioms, several chosen as near-synonyms of the correct answer to block shallow pattern matching; the model selects the idiom that fits the context, learned few-shot from 42 labelled training examples.\n",
  "metric": {
   "baseline_note": "14.3% is the paper's majority-class baseline, matching the seven-way random-guess rate. See the fewclue family page for the top overall few-shot method scores.\n",
   "direction": "higher_is_better",
   "human_baseline": 87.1,
   "max_score": 100,
   "name": "accuracy",
   "random_baseline": 14.3,
   "unit": "%"
  },
  "name": "FewCLUE: CHID (Chinese Idiom Cloze Test)",
  "page_kind": "subset",
  "paper": {
   "arxiv": "2107.07498",
   "title": "FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark",
   "url": "https://arxiv.org/abs/2107.07498",
   "year": 2021
  },
  "publisher": {
   "authors": [
    "Liang Xu",
    "Xiaojing Lu",
    "Chenyang Yuan",
    "Xuanwei Zhang",
    "Huilin Xu",
    "Hu Yuan",
    "Guoao Wei",
    "Xiang Pan",
    "Xin Tian",
    "Libo Qin",
    "Hu Hai"
   ],
   "org": "CLUE team",
   "url": "https://github.com/CLUEbenchmark/FewCLUE"
  },
  "released": "2021-04",
  "repo_url": "https://github.com/CLUEbenchmark/FewCLUE",
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark (Xu et al., arXiv:2107.07498)",
    "url": "https://arxiv.org/abs/2107.07498"
   },
   {
    "accessed": "2026-09-08",
    "title": "CLUEbenchmark/FewCLUE GitHub repository (task description, dataset statistics)",
    "url": "https://github.com/CLUEbenchmark/FewCLUE"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass FewCLUE_chid dataset config",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/FewCLUE_chid/FewCLUE_chid_gen.py"
   }
  ],
  "status": "unknown",
  "subcategory": "Chinese idiom cloze test (multiple-choice fill-in-the-blank), few-shot",
  "summary": "FewCLUE's Chinese idiom cloze task: pick the idiom that fits a masked slot from seven near-synonym candidates, learned from 42 labelled training examples.",
  "tags": [
   "chinese",
   "few-shot",
   "fewclue-subset",
   "cloze"
  ],
  "task_format": "Seven-way multiple-choice cloze, graded on the single correct candidate; evaluated from a 42-example few-shot training split (six examples for each of the seven blank positions), one of five parallel splits FewCLUE provides for this task.\n"
 }
}