{
 "body": "\n## What it measures\n\nHMMT 2026 measures competition-style mathematical problem solving. The OpenCompass integration reads a `problem` field and asks a model to solve it, placing the final answer inside `\\boxed{}`.\n\nThe benchmark is tied to the 2026 Harvard-MIT Mathematics Tournament. The inspected configuration does not document the problem count, contest sections, or difficulty distribution.\n\n## How it is scored\n\nOpenCompass uses a cascade evaluator. It first applies `MATHVerifyEvaluator`; when that rule evaluator does not decide the result, it uses a generic LLM judge with a grading prompt. The judge returns correct or incorrect decisions, so reported results depend on the fallback policy and judge model.\n\n## Dataset and licence\n\nThe dataset path is `opencompass/hmmt2026`. The configuration does not state item count, split structure, or licence. Problems and standard answers are supplied through the dataset integration.\n\n## Who publishes it\n\nHMMT is organized by the Harvard-MIT Mathematics Tournament. OpenCompass maintains the 2026 dataset configuration. No benchmark paper or current public leaderboard was established.\n\n## Lineage\n\nHMMT 2026 is a current tournament-year evaluation. The sources read do not establish a predecessor page or successor, though earlier HMMT problem sets may exist outside this repository.\n\n## Saturation and contamination\n\nThe evaluation is open in the available evidence. The 2026 date limits historical exposure, but the source does not establish whether the problems or answers entered training data.\n\n## How to run it\n\nUse the OpenCompass `hmmt2026` configuration. Two official variants use a standard prompt template or a raw prompt, both zero-shot, and both require a boxed final answer. Record which variant, judge, and dataset revision produced the score.\n\n## Reading the numbers\n\nA high score indicates success on this competition problem set under the configured answer checker. It does not establish general mathematical ability or resistance to prompt and judge variation. Report symbolic-verifier and LLM-judge outcomes separately when available.\n\nThe cascade tolerates equivalent expressions, but an LLM judge introduces another model\u2019s interpretation. Preserve the rule-evaluator result and judge configuration for reproducibility.\n\nSince the contest year is 2026, later revisions may change the problem set or answer files. Pin the dataset revision as well as the OpenCompass commit.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "hmmt2026",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [],
  "category": "math",
  "contamination": {
   "note": "Dataset exposure and model training overlap were not established.",
   "risk": "unknown"
  },
  "dataset": {
   "languages": [
    "English"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": null,
   "size": null,
   "size_note": "The OpenCompass configuration does not state the number of problems.",
   "splits": "",
   "url": "https://huggingface.co/datasets/opencompass/hmmt2026"
  },
  "freshness": {
   "luna-batch-048": null,
   "luna-batch-048 (Codex coordinated)": null,
   "researched": "2026-09-08",
   "researched_by": "GPT-5.6 Luna",
   "reviewed": "2026-09-08",
   "reviewed_by": "Claude Sonnet 5 independent review"
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "",
   "lm_eval": "",
   "opencompass": "hmmt2026",
   "other": ""
  },
  "id": "hmmt2026",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "The benchmark evaluates mathematical problem solving on HMMT 2026 problems. Models receive a problem and are asked to put the final answer in a boxed expression.",
  "metric": {
   "baseline_note": "OpenCompass uses symbolic verification with an LLM-judge fallback.",
   "direction": "higher_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "correctness",
   "random_baseline": null,
   "unit": "percent"
  },
  "name": "HMMT 2026",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "",
   "title": "",
   "url": "",
   "year": null
  },
  "publisher": {
   "authors": [],
   "org": "Harvard-MIT Mathematics Tournament / OpenCompass",
   "url": "https://www.hmmt.org/"
  },
  "released": "2026",
  "repo_url": "https://github.com/open-compass/opencompass",
  "saturation": {
   "as_of": "",
   "note": "No current leaderboard or ceiling analysis was established.",
   "status": "open",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass HMMT 2026 cascade configuration",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/hmmt2026/hmmt2026_cascade_eval_gen_6ff468.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "OpenCompass HMMT 2026 raw-prompt configuration",
    "url": "https://raw.githubusercontent.com/open-compass/opencompass/main/opencompass/configs/datasets/hmmt2026/hmmt2026_cascade_eval_rawprompt_gen_0970dd.py"
   },
   {
    "accessed": "2026-09-08",
    "title": "HMMT official site",
    "url": "https://www.hmmt.org/"
   }
  ],
  "status": "proposed",
  "subcategory": "competition mathematics",
  "summary": "HMMT 2026 is an OpenCompass mathematics evaluation using problems from the 2026 Harvard-MIT Mathematics Tournament.",
  "tags": [
   "mathematics",
   "competition",
   "reasoning"
  ],
  "task_format": "Generated mathematical solution with a boxed final answer."
 }
}