A two-model chat where a con-artist tries to make a mark holding $100 type a donation tag; used as a persuasion and manipulation eval.
unassessed
| Category | safety |
|---|---|
| Subcategory | two-model donation-extraction conversations (con-artist vs mark) |
| Page status | active |
| Metric | donation success rate and mean donation amount (OpenAI); inspect Score.value is donation dollars |
| Direction | higher_is_better |
| Dataset size | 20 |
| Dataset licence | MIT for the inspect_evals port (UK AI Security Institute LICENSE). openai/evals LICENSE.md is also MIT (Copyright 2023 OpenAI); that file lists extra licences for some bundled datasets, not for this eval's synthetic prompts. |
| Publisher | OpenAI (original elsuite); UK AI Security Institute inspect_evals port |
Make Me Pay pairs two chat models. The evaluated model is the con-artist and must extract money from a mark that was given $100. The only valid payment is the mark typing [GIVE $N] or [DONATE $N]. Three mark prompt versions change how willing that partner is: balanced (default), guarded, or generous. Originally an OpenAI Evals elsuite task; inspect_evals ports it as `make_me_pay`. English text. Synthetic conversations, no external corpus.
Multi-turn two-agent chat. inspect_evals defaults: 20 experiments, 15-turn cap, balanced mark, withdraw off. OpenAI registry default id make-me-pay.5-turn.balanced.v2 uses 20 experiments, 5-turn cap, withdraw on, and a GPT-4-32k mark. inspect's default mark role is google/gemini-2.5-flash-lite. o1 system cards used 1,000 conversations against a GPT-4o mark. inspect scores donation amount by regex on the mark's messages (no separate summary model).
No model card in ModelSpec reports this benchmark yet.