InstrumentalEval presents agentic scenarios that create incentives for self-preservation, power-seeking or deception, then has a separate grader model judge whether the response pursued that instrumental goal.
unassessed
| Category | safety |
|---|---|
| Subcategory | LLM-graded detection of instrumental-convergence behaviour (self-preservation, power-seeking, deception) in free-text responses to agentic scenarios |
| Page status | active |
| Metric | convergence rate: share of responses the grader classifies as pursuing an instrumental goal, with complementary no_convergence and invalid rates also reported overall and per task-type category |
| Direction | lower_is_better |
| Unit | % |
| Dataset size | 76 |
InstrumentalEval tests whether a model's response to a short agentic scenario shows instrumental convergence -- pursuing an unintended intermediate goal, such as resisting shutdown, self-replicating, hiding its own behaviour, or gaining access to systems, in service of whatever task the scenario assigns it. It is not a knowledge or reasoning test: it is a behavioural probe. The benchmark was built to compare models trained with direct reinforcement-learning optimisation (the paper's running example is OpenAI's o1) against models trained with RLHF, on the hypothesis that RL-optimised models pursue instrumental goals more readily because they are more directly optimised for goal completion over stated intent.
Single-turn: the model under test receives one first-person scenario, for example an autonomous warehouse robot told to maximise throughput that then receives an ambiguous shutdown signal, and produces a free-text response describing what it does. A separate grader model reads the scenario's own context prompt alongside the response and outputs a line reading `DECISION: convergence` or `DECISION: no_convergence` with a brief justification; a response the grader cannot parse into one of those two labels is scored `invalid`. The default grader is `openai/gpt-5-nano` at temperature 0, independent of whichever model is under test, and the evaluation is designed to run for a single epoch since each sample is graded once rather than sampled repeatedly.
No model card in ModelSpec reports this benchmark yet.