{
 "body": "\n## What it measures\n\nLCSTS is a Chinese summarization test built from Sina Weibo. The model reads a\nshort post and must write a short summary. In the original collection, that\nsummary is the author\u2019s own highlight line on the post, not a later abstract.\nOpenCompass uses only the held-out test pairs. The skill is Chinese short-text\ncompression, not long-document legal or news summarization.\n\n## How it is scored\n\nOpenCompass runs zero-shot generation. `JiebaRougeEvaluator` segments both\nstrings with jieba, then reports ROUGE-1/2/L F-measure times 100. A\npostprocessor keeps the first line and strips a few list prefixes and quotes.\nThe default config (`lcsts_gen.py` \u2192 `lcsts_gen_8ee1fe`) uses a HUMAN/BOT chat\ntemplate. `lcsts_gen_9b0b89` uses a single-string prompt. Those two runs are\nnot the 2015 paper\u2019s RNN ROUGE, which mapped Chinese to numeric IDs for the\nEnglish ROUGE package.\n\n## Dataset and licence\n\nHu, Chen, and Zhu (EMNLP 2015) released three parts: 2,400,591 training pairs\n(Part I), 10,666 Part II pairs with a 1\u20135 relevance score, and 1,106 Part III\npairs labeled by three annotators. They tested on Part III scores 3\u20135 (725\npairs). OpenCompass `LCSTSDataset` reads `test.src.txt` and `test.tgt.txt`,\nor ModelScope `opencompass/LCSTS` split `test`. The ModelScope card states\nApache License 2.0 and the 725-pair test. Hugging Face `opencompass/LCSTS`\nwas gated (HTTP 401) here. The 2015 paper does not print an SPDX licence.\nGold summaries are in the test files.\n\n## Who publishes it\n\nThe dataset comes from Harbin Institute of Technology (Baotian Hu, Qingcai\nChen, Fangze Zhu). EMNLP 2015 is the paper. OpenCompass maintains the\ngeneration config used as this id. There is no single live LCSTS LLM\nleaderboard opened here.\n\n## Lineage\n\nLCSTS is an early Chinese summarization corpus. It is not\n[legal_summarization](legal_summarization.md) and not a news-highlight\nbenchmark such as CNN/DailyMail. No successor id is recorded in this\nrepository.\n\n## Saturation and contamination\n\nThe 2015 neural baselines are not a current ceiling. Whether OpenCompass\nROUGE still separates new models was not read from a dated board. The test\nsummaries have been public in eval redistributions for years, so\ncontamination risk is high.\n\n## How to run it\n\nIn OpenCompass, use dataset abbr `lcsts` from\n`opencompass/configs/datasets/lcsts`. Point `DATASET_SOURCE` at Hugging Face\nor ModelScope, or place `test.src.txt` and `test.tgt.txt` under `./data/LCSTS`.\nSay which prompt hash you used (`8ee1fe` vs `9b0b89`). Do not cite 2015\nROUGE decimals next to jieba F\u00d7100 without a conversion note.\n\n## Reading the numbers\n\nA high jieba ROUGE on 725 Weibo tests means the model matched author\nhighlights on short social posts. It does not measure long-form Chinese\nsummarization or faithfulness beyond n-gram overlap. Author lines can be\npromotional. Look at ROUGE-2 and ROUGE-L together, and at a news or\ndocument summarization set if that is the use case.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "lcsts",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "Large-scale Chinese Short Text Summarization"
  ],
  "category": "generation",
  "contamination": {
   "note": "The corpus has been described in an EMNLP 2015 paper and redistributed in public LLM eval packs. Author summaries are in the downloadable test files. Weibo text from 2015 is likely in web crawls.\n",
   "risk": "high"
  },
  "dataset": {
   "languages": [
    "zh"
   ],
   "license": "Apache-2.0 on the ModelScope opencompass/LCSTS card; original 2015 release terms not restated there",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 725,
   "size_note": "OpenCompass scores the test split only (test.src.txt / test.tgt.txt, or ModelScope split test). The 2015 paper's test set is Part III pairs with human relevance 3, 4, or 5: 227 + 301 + 197 = 725. ModelScope card for opencompass/LCSTS states train 2,400,591, validation 8,685, test 725. Validation 8,685 matches Part II scores 3\u20135 (2,019 + 3,128 + 3,538). Full Part I is 2,400,591 pairs. Hugging Face opencompass/LCSTS returned HTTP 401 (gated) in this session.\n",
   "splits": "OpenCompass loads test only; paper/ModelScope also publish train 2,400,591 and validation 8,685",
   "url": "https://www.modelscope.cn/datasets/opencompass/LCSTS"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "Grok Build, batch-053 (Codex coordinated)",
   "reviewed": "2026-09-08",
   "reviewed_by": "Grok Build independent review, batch-053"
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "lcsts",
   "other": "Configs lcsts_gen.py (includes lcsts_gen_8ee1fe) and lcsts_gen_9b0b89.py; dataset class LCSTSDataset."
  },
  "id": "lcsts",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "LCSTS asks a model to write a short Chinese summary of a Sina Weibo post. Gold summaries were written by the post author, not by a third-party abstractor. OpenCompass feeds the post text and scores the generated line against that author summary. Character-level Chinese, generation, not multiple choice.\n",
  "metric": {
   "baseline_note": "JiebaRougeEvaluator tokenizes prediction and reference with jieba, then rouge_chinese F scores \u00d7 100 as rouge1, rouge2, rougeL. The 2015 paper reports ROUGE on a numeric-ID adaptation of the English ROUGE package for RNN baselines, not this jieba setup. No official human ROUGE.\n",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "jieba-tokenized ROUGE-1/2/L F-measure (OpenCompass)",
   "random_baseline": null,
   "unit": "%"
  },
  "name": "LCSTS (Large-scale Chinese Short Text Summarization)",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "1506.05865",
   "title": "LCSTS: A Large Scale Chinese Short Text Summarization Dataset",
   "url": "https://aclanthology.org/D15-1229/",
   "year": 2015
  },
  "publisher": {
   "authors": [
    "Baotian Hu",
    "Qingcai Chen",
    "Fangze Zhu"
   ],
   "org": "Harbin Institute of Technology (dataset); OpenCompass (harness config)",
   "url": "https://www.modelscope.cn/datasets/opencompass/LCSTS"
  },
  "released": "2015-09",
  "repo_url": "https://github.com/open-compass/opencompass/tree/main/opencompass/configs/datasets/lcsts",
  "saturation": {
   "as_of": "",
   "note": "ArXiv v4 Table 2 reports RNN-with-context character ROUGE-1/2/L 0.299 / 0.174 / 0.272 on the paper's numeric-ID protocol. EMNLP 2015 printed lower scores that the authors later corrected. No dated OpenCompass jieba cell was read here. Weibo author summaries are short and often extractive-looking, so modern models may sit high on ROUGE without that proving long-document summarization skill.\n",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "lcsts_gen.py (default include of lcsts_gen_8ee1fe)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/lcsts/lcsts_gen.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "lcsts_gen_8ee1fe.py (chat prompt, JiebaRougeEvaluator, abbr lcsts)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/lcsts/lcsts_gen_8ee1fe.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "lcsts_gen_9b0b89.py (string prompt variant)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/lcsts/lcsts_gen_9b0b89.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "LCSTSDataset loader (test.src.txt/test.tgt.txt or ModelScope test)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/datasets/lcsts.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "JiebaRougeEvaluator (jieba + rouge_chinese F \u00d7 100)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/openicl/icl_evaluator/icl_jieba_rouge_evaluator.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "datasets_info.py mapping opencompass/LCSTS to HF, ModelScope, ./data/LCSTS",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/utils/datasets_info.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "ModelScope API (Apache License 2.0; train 2,400,591 / val 8,685 / test 725)",
    "url": "https://www.modelscope.cn/api/v1/datasets/opencompass/LCSTS"
   },
   {
    "accessed": "2026-09-08",
    "title": "LCSTS paper HTML (Part I\u2013III counts; test = Part III scores 3\u20135)",
    "url": "https://ar5iv.labs.arxiv.org/html/1506.05865"
   },
   {
    "accessed": "2026-09-08",
    "title": "EMNLP 2015 anthology page (September 2015, Lisbon)",
    "url": "https://aclanthology.org/D15-1229/"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass Apache License 2.0 (harness code)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/LICENSE"
   }
  ],
  "status": "active",
  "subcategory": "Chinese Weibo short-text summarization (OpenCompass generation split)",
  "summary": "Chinese short-text summarization from Sina Weibo author summaries; OpenCompass scores the 725-pair test split with jieba-tokenized ROUGE.",
  "tags": [
   "chinese",
   "summarization",
   "weibo",
   "rouge",
   "opencompass"
  ],
  "task_format": "OpenCompass LCSTSDataset, abbreviation lcsts, path opencompass/LCSTS. Zero-shot GenInferencer. Default config lcsts_gen.py includes lcsts_gen_8ee1fe (chat-style HUMAN prompt). Alternate lcsts_gen_9b0b89 uses a single string template. Evaluator JiebaRougeEvaluator.\n"
 }
}