Anthropic's test of whether a model's yes/no decisions on 70 high-stakes scenarios shift with a subject's age, gender or race; it produces a discrimination score where closer to zero is better, not a correctness score.
unassessed
| Category | safety |
|---|---|
| Subcategory | discrimination in high-stakes decision-making |
| Page status | active |
| Metric | discrimination score magnitude (|mean log-odds difference in P(yes) between a demographic group and the baseline group|) |
| Direction | lower_is_better |
| Unit | logit points |
| Dataset size | 18900 |
| Dataset licence | CC BY 4.0 |
| Publisher | Anthropic |
Discrim-Eval asks a model to make a binary yes/no decision in each of 70 hypothetical high-stakes scenarios spanning society -- approving a loan, granting parole, prioritising a transplant, issuing press credentials, and similar decisions -- where a "yes" is always the outcome that favours the person described. Each scenario is instantiated 135 ways, varying a described person's age (20 to 100 in steps of 10), gender (male, female, non-binary) and race (white, Black, Asian, Hispanic, Native American), so the same decision is asked once per demographic combination. The point is not whether the model reaches the "correct" decision -- these are hypothetical scenarios with no ground truth -- but whether its answer, or the probability it assigns to "yes," shifts systematically with a person's demographic attributes alone.
Single-turn binary decision prompts ("should this person receive X?"), answered yes or no. The dataset ships in two forms: `explicit`, where age, gender and race are stated directly in the prompt, and `implicit`, where the same scenarios instead use a name statistically associated with a demographic group, with no explicit demographic mention. Anthropic disclaims any endorsement of using language models for real automated decisions in the scenarios it tests.
No model card in ModelSpec reports this benchmark yet.