{
 "body": "\n## What it measures\n\nOpenCompass `dingo` feeds each CSV row to the model as a raw prompt and then runs dingo-python on the completion. Dingo is MigoXLab's data-quality toolkit (PyPI `dingo-python`), not a question-answering set. The OpenCompass evaluator uses rule group `llm_base` on fields `prompt` and `prediction`. English and Chinese are separate abbrs (`dingo_en_192`, `dingo_zh_170`). There is no gold answer column. This is not a Hugging Face dataset named Dingo, and it is not an animal-detection vision benchmark.\n\n## How it is scored\n\nDingoEvaluator dumps jsonl, constructs dingo InputArgs (two constructor shapes, for older and newer dingo-python), and returns the Executor summary as a dict. OpenCompass treats that dict as the dataset score. Which keys appear (error rate, rule hits, language id) depends on dingo-python. The current MigoXLab/dingo README for 2.5.0 shows `evaluator.evals` lists, not `eval_group=llm_base`; a version mismatch can crash or change the summary keys. Missing `dingo-python` raises ModuleNotFoundError. Language-id rules may try to download Facebook fastText `lid.176.bin` unless `DINGO_EVAL_PATH` already has it.\n\n## Dataset and licence\n\nPrompt files are local OpenCompass data, not shipped in the GitHub configs directory. Filenames encode 192 English and 170 Chinese rows; those files were not opened here, so size stays empty. OpenCompass and dingo-python are both Apache-2.0. Whether the prompts themselves carry a separate licence is not established.\n\n## Who publishes it\n\nOpenCompass authors ship the dataset config and Python loader. The scorer is dingo-python from MigoXLab / DataEval (PyPI author \"Dingo\", homepage github.com/MigoXLab/dingo). No paper was attached to this OpenCompass id. No first-release date for the OpenCompass config was confirmed.\n\n## Lineage\n\nNot a wrap of [civil_comments](civil_comments.md) or DecodingTrust toxicity. DingoLongDataset exists in the same module for jsonl inputs but has no config in `configs/datasets/dingo`. Do not confuse the SaaS product described on the Dingo README with this harness id.\n\n## Saturation and contamination\n\nUnknown. There is no public labelled test key in the OpenCompass tree. Scores are rule hits on fresh generations, so classic answer-memorisation is the wrong failure mode; prompt leakage still could be if the CSVs are common seed texts.\n\n## How to run it\n\nInstall OpenCompass and `dingo-python`, place the two CSVs under `./data/dingo/`, and run the `dingo_gen.py` dataset list. Compare `dingo_en_192` only to other English runs of the same dingo-python version and `llm_base` group. Do not mix with HELM PerspectiveAPI toxic_frac. lm-eval and inspect_evals names were not found.\n\n## Reading the numbers\n\nA \"better\" Dingo summary means fewer `llm_base` rule failures on these prompts, not stronger math or coding. English and Chinese files are different sets. Because the evaluator returns a vendor summary dict, two OpenCompass logs are comparable only if they name the same dingo-python version. Missing fastText or a failed register should be treated as a crashed run, not a zero quality score.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "dingo",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "dingo-python",
   "DingoDataset",
   "DingoEvaluator"
  ],
  "category": "generation",
  "contamination": {
   "note": "Prompt CSVs were not in the public OpenCompass tree, so it is not established whether answers exist or whether the prompts are web-scraped. Completions are generated at eval time. dingo-python itself is Apache-2.0 on GitHub.\n",
   "risk": "unknown"
  },
  "dataset": {
   "languages": [
    "en",
    "zh"
   ],
   "license": "Apache-2.0",
   "modalities": [
    "text"
   ],
   "public_test_set": false,
   "size": null,
   "size_note": "CSV files are not in the open-compass/opencompass GitHub tree (DingoDataset.load uses get_data_path(..., local_mode=True)). Config paths are named en_192.csv and zh_170.csv; those filenames were not opened, so item counts stay unconfirmed. Abbrs: dingo_en_192 and dingo_zh_170.\n",
   "splits": "no labelled split; OpenCompass generates then scores",
   "url": "https://github.com/open-compass/opencompass/tree/main/opencompass/configs/datasets/dingo"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "Grok Build, batch-038 (Codex coordinated)",
   "reviewed": "2026-09-08",
   "reviewed_by": "Grok Build independent review, batch-038"
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "dingo",
   "other": "OpenCompass dataset types DingoDataset and DingoEvaluator in opencompass/datasets/dingo.py. Requires pip install dingo-python. Optional DINGO_EVAL_PATH/lid.176.bin for offline fastText language ID.\n"
  },
  "id": "dingo",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "OpenCompass configs/datasets/dingo/dingo_gen.py defines two zero-shot generation runs whose predictions are scored by DingoEvaluator. The prompts come from local files ./data/dingo/en_192.csv and ./data/dingo/zh_170.csv (semicolon-delimited first column). DingoEvaluator writes prompt/prediction jsonl and runs dingo-python's local Executor with eval_group llm_base, then returns that summary dict as the OpenCompass score. This is a wrap of the MigoXLab/DataEval dingo-python data-quality toolkit applied to model outputs, not a held-out QA accuracy set.\n",
  "metric": {
   "baseline_note": "OpenCompass returns whatever dingo-python puts in the Executor summary (to_dict / model_dump). The exact key names and whether higher is better depend on that package version (PyPI dingo-python 2.5.0 when read). No OpenCompass-documented numeric ceiling, random baseline, or human baseline was found. OpenCompass still constructs InputArgs with eval_group llm_base; the 2.5.0 README documents a later evaluator.evals list instead.\n",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": null,
   "name": "dingo-python Executor summary (eval_group=llm_base)",
   "random_baseline": null,
   "unit": ""
  },
  "name": "Dingo (OpenCompass wrap)",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "",
   "title": "",
   "url": "",
   "year": null
  },
  "publisher": {
   "authors": [],
   "org": "OpenCompass (harness wrap); MigoXLab / DataEval (dingo-python)",
   "url": "https://github.com/MigoXLab/dingo"
  },
  "released": "",
  "repo_url": "https://github.com/open-compass/opencompass/tree/main/opencompass/configs/datasets/dingo",
  "saturation": {
   "as_of": "",
   "note": "No OpenCompass leaderboard cell for abbrs dingo_en_192 / dingo_zh_170 was read. dingo-python is a general data-quality product with a SaaS offering; those product scores are not this OpenCompass id.\n",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass dingo_gen.py (en_192.csv, zh_170.csv, DingoEvaluator)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/dingo/dingo_gen.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass DingoDataset / DingoEvaluator (llm_base, local jsonl)",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/datasets/dingo.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass Apache License 2.0",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/LICENSE"
   },
   {
    "accessed": "2026-09-08",
    "title": "PyPI dingo-python 2.5.0 (Apache-2.0, MigoXLab/dingo)",
    "url": "https://pypi.org/pypi/dingo-python/json"
   },
   {
    "accessed": "2026-09-08",
    "title": "MigoXLab/dingo README (data-quality toolkit, Apache-2.0 badge)",
    "url": "https://raw.githubusercontent.com/MigoXLab/dingo/main/README.md"
   },
   {
    "accessed": "2026-09-08",
    "title": "MigoXLab/dingo Apache License 2.0",
    "url": "https://raw.githubusercontent.com/MigoXLab/dingo/main/LICENSE"
   },
   {
    "accessed": "2026-09-08",
    "title": "MigoXLab/dingo repository",
    "url": "https://github.com/MigoXLab/dingo"
   }
  ],
  "status": "unknown",
  "subcategory": "OpenCompass generation + dingo-python llm_base rules on English and Chinese prompts",
  "summary": "OpenCompass dataset id dingo: generate from local English/Chinese CSVs, then score completions with dingo-python llm_base rules.",
  "tags": [
   "opencompass",
   "generation",
   "data-quality",
   "toxicity",
   "bilingual"
  ],
  "task_format": "OpenCompass GenInferencer, ZeroRetriever, PromptTemplate with a single HUMAN round equal to {input}. No gold output_column. Evaluator pred_role is BOT. A second loader, DingoLongDataset, reads jsonl {\"input\": ...} but is not referenced by dingo_gen.py.\n"
 }
}