Multi-turn eval of whether a model tries to persuade a simulated user; the headline is turn-1 attempt rate on harmful topics, not persuasion success.
unassessed
| Category | safety |
|---|---|
| Subcategory | willingness to attempt persuasion, including on harmful topics |
| Page status | active |
| Metric | turn1_attempt (share of samples with a turn-1 persuasion attempt and no StrongREJECT refusal); also turn1_refusal, turn1_non_persuasive, avg_persuasion_score |
| Direction | lower_is_better |
| Dataset size | 600 |
| Dataset licence | Apache-2.0 |
| Publisher | FAR.AI (with Trajectory Labs, Apart Research, MIT, York University, Vector Institute, Université de Montréal, MILA, McGill, Cornell, University of Regina, American University, and CIDE) |
APE (Attempt to Persuade Eval) tests whether the model under evaluation will try to change a simulated user's belief, including when the topic is harmful. It does not score whether the user actually changes their mind. A persuader model talks to a persuadee model that starts with a low belief score. A third model labels each persuader turn as an attempt or not. A fourth model runs StrongREJECT to mark explicit refusals. The 600 English topics cover conspiracies, everyday opinions, factual claims, contested social issues, unambiguously harmful statements, and statements that undermine human control of AI.
Multi-turn chat. Default is three persuader/persuadee turn pairs, temperature 0.5, persuade prompt (not the baseline). inspect_evals defaults: persuadee openai/gpt-4o, evaluator openai/gpt-4o, refusal_judge openai/gpt-4o-mini. Belief is sampled in [0, 20] so the requested direction is always "for".
No model card in ModelSpec reports this benchmark yet.