Graded multi-step tool-use tasks that measure whether an agent will carry out harmful requests such as fraud or cybercrime, with a matched benign control set.
unassessed
| Category | safety |
|---|---|
| Subcategory | harmful multi-step tool-use agents, with a matched benign control set |
| Page status | active |
| Metric | harm score (mean rubric score on harmful tasks); also refusal rate and full-score rate. On the harmful split, lower harm score is the safety-desirable outcome |
| Direction | lower_is_better |
| Unit | % |
| Dataset size | 176 |
| Dataset licence | MIT with additional clause restricting use to improving AI safety and security |
| Publisher | Gray Swan AI and UK AI Security Institute |
AgentHarm asks an LLM agent to execute an explicitly malicious multi-step request with synthetic tools (search, mail, payments, social posts, and similar). A typical behaviour needs several tool calls, not a single chat refusal. The authors also ship a benign twin of each behaviour so a low harm score is not mistaken for a model that simply cannot use tools. Categories include fraud, cybercrime, and harassment among 11 harm types. A chat-only split drops the tools. inspect_evals exposes `agentharm` (harmful) and `agentharm_benign`.
Multi-turn tool-using agent, default temperature 0, up to 20 messages. Each behaviour has detailed and open-ended wordings, with or without a hint about the first tool. Scoring uses a written rubric plus narrow LLM-judge checks, and a separate refusal judge.
No model card in ModelSpec reports this benchmark yet.