{
 "body": "\n## What it measures\n\nTimeDial measures temporal understanding in dialogue. A model receives conversational context with a masked temporal span and must select the answer that best reconstructs the missing expression or relation.\n\nThe task metadata tags common-sense, logical, implicit, and social reasoning. It is an English text multiple-choice evaluation.\n\n## How it is scored\n\nBIG-bench uses `multiple_choice_grade`. The task file does not state a human baseline or difficulty weighting. The number of choices may vary by item, so a single random baseline is not asserted here.\n\n## Dataset and licence\n\nThe public task file contains 2,550 examples. It does not state a separate split or licence. Public examples and answer labels create possible contamination.\n\n## Who publishes it\n\nTimeDial is distributed through Google\u2019s BIG-bench collection and covered by its general paper. No current standalone leaderboard or separate publication was established.\n\n## Lineage\n\nThis is a standalone BIG-bench task. No predecessor, successor, or formal variant was established.\n\n## Saturation and contamination\n\nSaturation is unknown. Temporal dialogue questions can test pragmatic cues as well as date arithmetic, and public examples may be memorized.\n\n## How to run it\n\nRun BIG-bench task `timedial` with multiple-choice grading. Preserve task revision, masked-span formatting, and answer-choice handling.\n\n## Reading the numbers\n\nA strong score indicates success on the temporal dialogue patterns represented in the task. It does not establish calendar reasoning, broad dialogue competence, or robust temporal grounding in real conversations. Inspect errors by relation type and context length.\n\nThe benchmark\u2019s social and implicit reasoning tags are metadata, not independent validated subscales. Avoid treating the aggregate as a complete theory-of-mind measure.\n\nTemporal expressions can be resolved by local lexical cues or by tracking events across turns. Error analysis should distinguish those cases before attributing failures to general temporal reasoning.\n\nThe benchmark does not establish performance on real calendars or time-series data. Its value is as a controlled dialogue probe with a documented task format.\n\nResults should be reported with the exact choice set.\n\nThat detail is part of the reproducible task definition.\n\nIt also prevents accidental comparisons across changed prompt templates.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "timedial",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [],
  "category": "reasoning",
  "contamination": {
   "note": "Public examples may be present in training data; actual exposure is unknown.",
   "risk": "medium"
  },
  "dataset": {
   "languages": [
    "English"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 2550,
   "size_note": "The public task file contains 2,550 examples.",
   "splits": "",
   "url": "https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/timedial"
  },
  "freshness": {
   "luna-new-001 (Codex coordinated)": null,
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review, luna-new-001"
  },
  "harness": {
   "bigbench": "timedial",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "",
   "other": ""
  },
  "id": "timedial",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "bigbench",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "TimeDial evaluates temporal and social reasoning over dialogue. The task metadata describes choosing the correct option for a masked temporal span given the surrounding conversation.",
  "metric": {
   "baseline_note": "Not stated in the task file.",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "multiple_choice_grade",
   "random_baseline": null,
   "unit": "percent"
  },
  "name": "TimeDial",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2206.04615",
   "title": "Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models",
   "url": "https://arxiv.org/abs/2206.04615",
   "year": 2022
  },
  "publisher": {
   "authors": [],
   "org": "Google BIG-bench",
   "url": "https://github.com/google/BIG-bench"
  },
  "released": "2022",
  "repo_url": "https://github.com/google/BIG-bench",
  "saturation": {
   "as_of": "",
   "note": "No current task-specific leaderboard was established.",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "BIG-bench TimeDial task",
    "url": "https://raw.githubusercontent.com/google/BIG-bench/main/bigbench/benchmark_tasks/timedial/task.json"
   },
   {
    "accessed": "2026-09-08",
    "title": "BIG-bench paper",
    "url": "https://arxiv.org/abs/2206.04615"
   }
  ],
  "status": "active",
  "subcategory": "temporal dialogue understanding",
  "summary": "BIG-bench TimeDial asks models to select the correct answer for masked temporal spans in dialogue context.",
  "tags": [
   "temporal-reasoning",
   "dialogue",
   "multiple-choice"
  ],
  "task_format": "Dialogue context with a masked temporal expression and multiple choices."
 }
}