{
 "body": "\nPart of the [FewCLUE](fewclue.md) family.\n\n## What it measures\n\nCLUEWSC is the CLUE benchmark's Chinese Winograd Schema Challenge, resampled into FewCLUE's few-shot\nformat: given a sentence with a marked pronoun and a marked noun phrase, the model judges true or\nfalse whether the pronoun refers to that noun phrase. Sentences are hand-picked from 36 contemporary\nChinese literary works and hand-annotated by linguists, so resolving them typically requires\nreal-world or commonsense inference about the scene being described rather than surface pattern\nmatching. CLUEWSC is one of six FewCLUE tasks that originate in the CLUE benchmark rather than being\nbuilt new for FewCLUE.\n\n## Reading the numbers\n\nThe labelled public test set holds 976 items, scored by accuracy against a 50% two-way baseline.\nThis task produced the single largest human/model gap in the FewCLUE paper: humans scored 98.0%,\nclose to ceiling, while every few-shot method tested landed between about 53% and 58.7% -- barely\nabove random guessing. That makes a high CLUEWSC score genuinely informative about coreference and\ncommonsense ability, precisely because the task has proven so hard for models to game; treat any\nscore well above the high-50s as notable and worth checking against the specific split used. See the\n[FewCLUE](fewclue.md) family page for the shared few-shot protocol and contamination notes that apply\nhere too.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "fewclue_cluewsc",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "category": "reasoning",
  "dataset": {
   "languages": [
    "zh"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 976,
   "size_note": "976 labelled public test items (test_public.json, used for scoring), plus 290 in the private test set (original leaderboard only), 32 train and 32 dev examples per split (5 parallel splits, train_0..train_4 / dev_0..dev_4, plus a merged train_few_all/dev_few_all), and no unlabelled data for this task (0, per the paper's Table 1, matching the GitHub README). Sentences are hand-picked from 36 contemporary Chinese literary works and hand-annotated by linguists.\n",
   "splits": "train_0..train_4 (32 each) + train_few_all; dev_0..dev_4 (32 each) + dev_few_all; test_public (976, labelled); test (290, private); unlabeled (0)",
   "url": "https://github.com/CLUEbenchmark/FewCLUE/tree/main/datasets/cluewsc"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "sonnet-5 agent, batch 5, slice B"
  },
  "harness": {
   "opencompass": "FewCLUE_cluewsc"
  },
  "id": "fewclue_cluewsc",
  "leaderboard_url": "https://www.cluebenchmarks.com/fewclue.html",
  "lineage": {
   "family": "fewclue"
  },
  "measures": "A Chinese sentence with a marked pronoun and a marked noun phrase, hand-picked from contemporary literary works; the model judges true or false whether the pronoun refers to that noun phrase, learned few-shot from 32 labelled training examples.\n",
  "metric": {
   "baseline_note": "50.0% is the paper's majority-class baseline, matching the two-way random-guess rate. See the fewclue family page for the top overall few-shot method scores.\n",
   "direction": "higher_is_better",
   "human_baseline": 98.0,
   "max_score": 100,
   "name": "accuracy",
   "random_baseline": 50.0,
   "unit": "%"
  },
  "name": "FewCLUE: CLUEWSC (Winograd Schema Coreference)",
  "page_kind": "subset",
  "paper": {
   "arxiv": "2107.07498",
   "title": "FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark",
   "url": "https://arxiv.org/abs/2107.07498",
   "year": 2021
  },
  "publisher": {
   "authors": [
    "Liang Xu",
    "Xiaojing Lu",
    "Chenyang Yuan",
    "Xuanwei Zhang",
    "Huilin Xu",
    "Hu Yuan",
    "Guoao Wei",
    "Xiang Pan",
    "Xin Tian",
    "Libo Qin",
    "Hu Hai"
   ],
   "org": "CLUE team",
   "url": "https://github.com/CLUEbenchmark/FewCLUE"
  },
  "released": "2021-04",
  "repo_url": "https://github.com/CLUEbenchmark/FewCLUE",
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark (Xu et al., arXiv:2107.07498)",
    "url": "https://arxiv.org/abs/2107.07498"
   },
   {
    "accessed": "2026-09-08",
    "title": "CLUEbenchmark/FewCLUE GitHub repository (task description, dataset statistics)",
    "url": "https://github.com/CLUEbenchmark/FewCLUE"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass FewCLUE_cluewsc dataset config",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/FewCLUE_cluewsc/FewCLUE_cluewsc_gen.py"
   }
  ],
  "status": "unknown",
  "subcategory": "Chinese Winograd Schema pronoun coreference resolution, few-shot",
  "summary": "FewCLUE's Winograd Schema task: judge whether a marked pronoun refers to a marked noun phrase in a Chinese sentence, learned from 32 labelled training examples.",
  "tags": [
   "chinese",
   "few-shot",
   "fewclue-subset",
   "coreference"
  ],
  "task_format": "Binary coreference judgment (true / false), graded on the single correct label; evaluated from a 32-example few-shot training split, one of five parallel splits FewCLUE provides for this task.\n"
 }
}