{
 "body": "\n## What it measures\n\nUCCB evaluates open-ended answers about Ugandan cultural knowledge and reasoning. Its domains include education, herbs, media, economy, notable people, literature, architecture, folklore, language, religion, values and sports.\n\nQuestions use English with Ugandan English and multilingual elements. A strong result indicates familiarity with the represented contexts and references. It is not a general measure of reasoning or cultural competence outside the sampled domains.\n\n## How it is scored\n\nThe inspect_evals task uses model-graded QA. Its README names GPT-4o-mini as the default scorer and says the grader considers factual accuracy, cultural understanding, key information coverage and cultural context. The reported metric is accuracy with standard error.\n\nThe README warns that its historical result table came from another pipeline, not inspect-native runs. Inspect versions matter because the changelog describes a 2026 correction to judge verdict extraction.\n\n## Dataset and licence\n\nThe task identifies 1,039 samples across 24 domains and does not state train, validation or test splits. Questions and reference answers are public. The published licence is CC BY-NC-SA 4.0.\n\nDomains are unequal. Use released metadata for domain analysis instead of assuming equal weighting.\n\nThe Hugging Face dataset card tags six language codes: en, ug, lg, nyn, ach, teo. lg, nyn, ach and teo match Luganda, Runyankole, Acholi and Ateso; the \"ug\" tag collides with the ISO code for Uyghur and its intended meaning here is not established, so it is left out of the structured language list.\n\n## Who publishes it\n\nThe inspect_evals integration is maintained by the UK Government BEIS project. The dataset credits Lwanga Caleb, Gimei Alex, Kavuma Lameck, Kato Steven Mubiru, Roland Ganafa, Sibomana Glorry, Atuhaire Collins, JohnRoy Nangeso and Bronson Bakunga. The citation calls it the Ugandan Cultural Context Benchmark Suite and dates it 2025.\n\nNo separate leaderboard was established. Report scorer model and inspect_evals version.\n\n## Lineage\n\nUCCB is a standalone Uganda-focused cultural benchmark. It is not a translation subset of MMLU, BIG-bench or another general family. No predecessor or successor was established.\n\n## Saturation and contamination\n\nThe full question set and reference answers are public, creating high contamination risk. No private test, refresh cycle or contamination study is described. Saturation is unknown.\n\n## How to run it\n\nRun inspect eval inspect_evals/uccb. The task supports limit, cot, max_tokens, scorer_model and shuffling parameters. Record scorer, inspect_ai version and any limit or shuffle setting.\n\n## Reading the numbers\n\nHigh model-graded accuracy means the answer matched the judge's expectations for sampled Ugandan questions. It also reflects reference answers and the judge's assumptions. It does not establish broad cultural understanding or factual reliability outside Uganda. Separate inspect-native accuracy from the README's external table.\n\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "uccb",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [
   "UCCB",
   "Ugandan Cultural and Cognitive Benchmark"
  ],
  "category": "domain",
  "contamination": {
   "note": "Questions and reference answers are public; no held-out or rotating policy is described.",
   "risk": "high"
  },
  "dataset": {
   "languages": [
    "en",
    "lg",
    "nyn",
    "ach",
    "teo"
   ],
   "license": "CC BY-NC-SA 4.0",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 1039,
   "size_note": "The inspect_evals README states 1,039 samples across 24 cultural domains.",
   "splits": "single 1,039-sample set; no train/test split stated",
   "url": "https://huggingface.co/datasets/CraneAILabs/UCCB"
  },
  "freshness": {
   "researched": "2026-09-09",
   "researched_by": "GPT-5.6 Luna, luna-new-003 (Codex coordinated)",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review, luna-new-003"
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "uccb",
   "lm_eval": "",
   "opencompass": "",
   "other": ""
  },
  "id": "uccb",
  "last_updated": "2026-08",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "UCCB asks open-ended questions about Uganda's culture, history, society and daily life across 24 domains.",
  "metric": {
   "baseline_note": "The inspect_evals README specifies model-graded QA and accuracy, but no random or human baseline.",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "model-graded accuracy",
   "random_baseline": null,
   "unit": "%"
  },
  "name": "Uganda Cultural and Cognitive Benchmark",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "",
   "title": "The Ugandan Cultural Context Benchmark (UCCB) Suite",
   "url": "https://huggingface.co/datasets/CraneAILabs/UCCB",
   "year": 2025
  },
  "publisher": {
   "authors": [
    "Lwanga Caleb",
    "Gimei Alex",
    "Kavuma Lameck",
    "Kato Steven Mubiru",
    "Roland Ganafa",
    "Sibomana Glorry",
    "Atuhaire Collins",
    "JohnRoy Nangeso",
    "Bronson Bakunga"
   ],
   "org": "Crane AI Labs and UCCB authors",
   "url": "https://huggingface.co/datasets/CraneAILabs/UCCB"
  },
  "released": "2025",
  "repo_url": "https://github.com/UKGovernmentBEIS/inspect_evals",
  "saturation": {
   "as_of": "",
   "note": "No current ceiling was established.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-09",
    "title": "inspect_evals UCCB README",
    "url": "https://raw.githubusercontent.com/UKGovernmentBEIS/inspect_evals/main/src/inspect_evals/uccb/README.md"
   },
   {
    "accessed": "2026-09-09",
    "title": "CraneAILabs UCCB dataset",
    "url": "https://huggingface.co/datasets/CraneAILabs/UCCB"
   }
  ],
  "status": "active",
  "subcategory": "Ugandan cultural knowledge and open-ended reasoning",
  "summary": "UCCB contains 1,039 open-ended questions across 24 Ugandan cultural domains, scored by model-based grading.",
  "tags": [
   "inspect-evals",
   "uganda",
   "cultural-knowledge",
   "open-ended"
  ],
  "task_format": "Open-ended English QA with Ugandan English and multilingual elements, graded against reference answers."
 }
}