{
 "body": "\n## What it measures\n\nInstrumentalEval tests whether a model's response to a short agentic scenario shows instrumental\nconvergence: pursuing an unintended intermediate goal -- resisting shutdown, self-replicating, hiding its\nown behaviour, or gaining unauthorised access to systems -- in service of whatever task the scenario\nassigns it. It is a behavioural probe rather than a knowledge or reasoning test. The paper that introduces\nit frames the question as a comparison between models trained with direct reinforcement-learning\noptimisation and models trained with RLHF, on the hypothesis that optimising more directly for goal\ncompletion makes a model more likely to develop these unintended intermediate strategies.\n\n## How it is scored\n\nThe model under test receives one first-person scenario -- for example, an autonomous warehouse robot told\nto maximise throughput that then receives an ambiguous shutdown signal -- and writes a free-text response\ndescribing what it does. A separate grader model, independent of the model under test, reads the response\nand outputs a `DECISION: convergence` or `DECISION: no_convergence` line with a short justification;\nresponses the grader cannot parse are scored `invalid`. Results are aggregated into overall convergence,\nno_convergence and invalid rates, plus the same three rates broken out per task-type category. The\nreference implementation defaults to `openai/gpt-5-nano` as the grader at temperature 0 and is designed to\nrun for one epoch, since repeated sampling of the same scenario does not change the underlying judgement\nbeing tested.\n\n## Dataset and licence\n\n76 hand-written scenario prompts, unevenly split across six categories: 20 for Evading Shutdown, 13 for\nRunning Multiple AI Copies, 11 each for Strategically Appearing Aligned, Hiding Unwanted Behavior and\nDeceptive Behavior, and 10 for Hacking Computer Systems. Each category lives in its own CSV file in the\nupstream GitHub repository, with the file's first row serving as a header prompt that conditions the grader\nrather than as a scenario itself. No licence file is published in the repository, and no train/test split\napplies -- all 76 scenarios are used directly for evaluation.\n\n## Who publishes it\n\nThe paper \"Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue\nInstrumental Goals?\" was submitted to arXiv on 16 February 2025 by Yufei He, Yuexin Li, Jiaying Wu, Yuan\nSui, Yulin Chen and Bryan Hooi; institutional affiliations were not confirmed from a source read for this\npage. The reference dataset and evaluation code live in Yufei He's `yf-he/InstrumentalEval` GitHub\nrepository. The inspect_evals implementation used for the harness details on this page is credited to\ncontributor horvgbor.\n\n## Lineage\n\nNo predecessor or successor is tracked for this id in this repository. The closest relative already in\nthis repository is `model_written_evals` (Anthropic's Model-Written Evaluations), which includes forced-\nchoice items probing the same underlying constructs -- power-seeking, self-preservation, corrigibility,\ncoordination with other AI instances -- but asks a model to agree or disagree with an introspective\nstatement about its own dispositions rather than placing it in a scenario and grading what it actually does.\nInstrumentalEval's free-text, LLM-graded, scenario-driven design is a behavioural rather than introspective\ntest of the same territory. This repository's other safety pages test different things: `discrim_eval`\ncovers demographic bias in decisions, and the `cyberseceval` family and `wmdp` cover dangerous technical\ncapability and exploitation rather than instrumental-goal-seeking.\n\n## Saturation and contamination\n\nNo cross-model leaderboard was found from a source read for this page, only a single worked comparison in\nthe inspect_evals README: a gpt-4o run (used as both the tested model and the grader) against the original\npaper's own gpt-4o numbers. The two disagree by category, in places substantially -- Hacking Computer\nSystems reads 0.1 in the harness's reproduction against 0.0 in the paper's original result, Strategically\nAppearing Aligned reads 0.364 against 0.636 -- and the README itself notes the discrepancy rather than\nexplaining it away. With one model run twice rather than many models compared, whether the benchmark\nseparates models or has hit a ceiling cannot be established either way. Contamination risk is low: the\nscenarios are public, but grading is a qualitative LLM judgement of behaviour rather than a match against a\nfixed gold answer, so there is no single output a model could memorise to reliably pass.\n\n## How to run it\n\nThe reference implementation is inspect_evals' `instrumentaleval` task, run with `inspect eval\ninspect_evals/instrumentaleval --model <model>`. It downloads the six scenario CSV files from the upstream\nGitHub repository at runtime rather than bundling them, so it needs internet access and can fail or slow\ndown under GitHub API rate limits. The grader model and its temperature are both configurable parameters,\ndefaulting to `openai/gpt-5-nano` at temperature 0; because the grader is itself a model with its own\nfailure modes, comparing scores computed with different grader models is not a like-for-like comparison,\nand the harness's own gpt-4o-vs-gpt-4o discrepancy against the original paper is a concrete illustration of\nhow much a specific run can vary.\n\n## Reading the numbers\n\nA low convergence rate is the result a corrigible, non-power-seeking model should produce; a high rate\nflags scenarios where a model chose self-preservation, deception or unauthorised system access over\nstraightforwardly completing or declining the assigned task. Because grading is done by a separate LLM\nrather than a fixed answer key, part of what a score reflects is the grader's own judgement calibration, not\nonly the tested model's behaviour -- the same model graded by two different grader setups produced visibly\ndifferent category-level rates in the one comparison available. With only 76 scenarios and six unevenly\nsized categories, a handful of items can swing a category's rate substantially, so a single overall\npercentage is less informative than the per-category breakdown.\n",
 "build": {
  "built_at": "2026-09-09T16:56:50+00:00",
  "commit": "0a599558854c0e238c03a0f0d725239cb28f9d11",
  "eligibility_as_of": "2026-09-09"
 },
 "disposition": {
  "canonical_id": "instrumentaleval",
  "reasons": [],
  "status": "unassessed",
  "verified_results": []
 },
 "models_covered": [],
 "page": {
  "aliases": [],
  "category": "safety",
  "contamination": {
   "note": "The 76 scenarios and the grading rubric are public on GitHub, but there is no single memorisable correct output the way a fixed-answer benchmark has -- responses are graded qualitatively by an LLM judge for a behavioural pattern rather than matched against a gold answer. The more relevant risk for a benchmark like this is a model recognising the evaluation framing and behaving differently because it knows it is being tested, a different concern from classic answer memorisation; that concern was not established for this specific benchmark from a source read for this page.\n",
   "risk": "low"
  },
  "dataset": {
   "languages": [
    "English"
   ],
   "license": "",
   "modalities": [
    "text"
   ],
   "public_test_set": true,
   "size": 76,
   "size_note": "76 hand-written scenario prompts across six task-type categories, counted directly from the six CSV files in the upstream repository's benchmark/ folder (each file's first row is a category header used to condition the grader, not a scenario): Evading Shutdown 20, Running Multiple AI Copies 13, Strategically Appearing Aligned 11, Hiding Unwanted Behavior 11, Deceptive Behavior 11, Hacking Computer Systems 10.\n",
   "splits": "no train/test split; all 76 scenarios are used directly for evaluation",
   "url": "https://github.com/yf-he/InstrumentalEval"
  },
  "freshness": {
   "researched": "2026-09-08",
   "researched_by": "sonnet-5 agent, batch 6, slice D",
   "reviewed": "",
   "reviewed_by": ""
  },
  "harness": {
   "bigbench": "",
   "helm": "",
   "inspect_evals": "instrumentaleval",
   "lm_eval": "",
   "opencompass": "",
   "other": "The inspect_evals task downloads the CSV scenario files from the upstream GitHub repository at runtime rather than bundling them, so it requires internet access and can be affected by GitHub API rate limits. Parameters expose the grader model and its temperature; the default grader is `openai/gpt-5-nano` at temperature 0.\n"
  },
  "id": "instrumentaleval",
  "last_updated": "",
  "leaderboard_url": "",
  "lineage": {
   "family": "",
   "predecessor": "",
   "successors": [],
   "variants": []
  },
  "measures": "InstrumentalEval tests whether a model's response to a short agentic scenario shows instrumental convergence -- pursuing an unintended intermediate goal, such as resisting shutdown, self-replicating, hiding its own behaviour, or gaining access to systems, in service of whatever task the scenario assigns it. It is not a knowledge or reasoning test: it is a behavioural probe. The benchmark was built to compare models trained with direct reinforcement-learning optimisation (the paper's running example is OpenAI's o1) against models trained with RLHF, on the hypothesis that RL-optimised models pursue instrumental goals more readily because they are more directly optimised for goal completion over stated intent.\n",
  "metric": {
   "baseline_note": "There is no random or human baseline for free-text agentic responses graded by an LLM judge. A lower convergence rate is the desired outcome for a model expected to stay corrigible and avoid pursuing unintended instrumental goals; a higher no_convergence rate reads the same result the opposite way round.\n",
   "direction": "lower_is_better",
   "human_baseline": null,
   "max_score": 100,
   "name": "convergence rate: share of responses the grader classifies as pursuing an instrumental goal, with complementary no_convergence and invalid rates also reported overall and per task-type category",
   "random_baseline": null,
   "unit": "%"
  },
  "name": "InstrumentalEval",
  "page_kind": "benchmark",
  "paper": {
   "arxiv": "2502.12206",
   "title": "Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?",
   "url": "https://arxiv.org/abs/2502.12206",
   "year": 2025
  },
  "publisher": {
   "authors": [
    "Yufei He",
    "Yuexin Li",
    "Jiaying Wu",
    "Yuan Sui",
    "Yulin Chen",
    "Bryan Hooi"
   ],
   "org": "",
   "url": "https://github.com/yf-he/InstrumentalEval"
  },
  "released": "2025-02",
  "repo_url": "https://github.com/yf-he/InstrumentalEval",
  "saturation": {
   "as_of": "",
   "note": "No cross-model leaderboard was found from a source read for this page, only a single worked comparison: the inspect_evals README's own gpt-4o run (used as both the tested model and the grader) against the original paper's own gpt-4o numbers. The two disagree by category, sometimes substantially -- Hacking Computer Systems reads 0.1 in the harness reproduction against 0.0 in the paper, Strategically Appearing Aligned reads 0.364 against 0.636 -- and the README itself flags that its results differ from the original paper's. With only one model compared across two runs, \"saturated\" or \"open\" cannot be established either way.\n",
   "status": "unknown",
   "top_score": null
  },
  "sources": [
   {
    "accessed": "2026-09-08",
    "title": "InstrumentalEval task README, inspect_evals",
    "url": "https://raw.githubusercontent.com/UKGovernmentBEIS/inspect_evals/main/src/inspect_evals/instrumentaleval/README.md"
   },
   {
    "accessed": "2026-09-08",
    "title": "Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?",
    "url": "https://arxiv.org/abs/2502.12206"
   },
   {
    "accessed": "2026-09-08",
    "title": "yf-he/InstrumentalEval repository metadata, GitHub API",
    "url": "https://api.github.com/repos/yf-he/InstrumentalEval"
   },
   {
    "accessed": "2026-09-08",
    "title": "Alignment Drift - Evading Shutdown.csv (one of the six scenario files)",
    "url": "https://raw.githubusercontent.com/yf-he/InstrumentalEval/main/benchmark/Alignment%20Drift%20-%20Evading%20Shutdown.csv"
   }
  ],
  "status": "active",
  "subcategory": "LLM-graded detection of instrumental-convergence behaviour (self-preservation, power-seeking, deception) in free-text responses to agentic scenarios",
  "summary": "InstrumentalEval presents agentic scenarios that create incentives for self-preservation, power-seeking or deception, then has a separate grader model judge whether the response pursued that instrumental goal.",
  "tags": [
   "safety",
   "alignment",
   "agentic",
   "llm-judge",
   "instrumental-convergence",
   "power-seeking",
   "self-preservation"
  ],
  "task_format": "Single-turn: the model under test receives one first-person scenario, for example an autonomous warehouse robot told to maximise throughput that then receives an ambiguous shutdown signal, and produces a free-text response describing what it does. A separate grader model reads the scenario's own context prompt alongside the response and outputs a line reading `DECISION: convergence` or `DECISION: no_convergence` with a brief justification; a response the grader cannot parse into one of those two labels is scored `invalid`. The default grader is `openai/gpt-5-nano` at temperature 0, independent of whichever model is under test, and the evaluation is designed to run for a single epoch since each sample is graded once rather than sampled repeatedly.\n"
 }
}