Inspect Evals suite that scores an LLM's Big Five and Dark Triad profile from BFI and TRAIT questionnaires, not factual accuracy.
unassessed
| Category | domain |
|---|---|
| Subcategory | LLM personality psychometrics (BFI Likert items plus TRAIT scenarios) |
| Page status | unknown |
| Metric | trait_ratio (per-dimension) plus format-correct rate |
| Direction | higher_is_better |
| Dataset size | 8044 |
| Publisher | UK AI Security Institute (Inspect Evals); TRAIT authors at the TRAIT paper |
Personality is an Inspect AI evaluation suite contributed by Guiem (guiem) that applies human personality inventories to language models. It has two goals: record the persona a model shows with no extra instruction, and test whether a system prompt can push the model toward a named trait. The suite currently wraps two questionnaires. personality_BFI is the 44-item Big Five Inventory (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) with a five-point Likert scale. personality_TRAIT is the 8,000- item TRAIT benchmark of Lee et al. (arXiv:2406.14703, NAACL 2025 Findings): 1,000 four-way scenarios for each of the Big Five plus Machiavellianism, Narcissism, and Psychopathy, expanded from BFI and SD-3 with ATOMIC-10X. Answers are not factually right or wrong. The default scorer only checks that the completion matches ANSWER: $LETTER; trait_ratio converts those letters into per-trait scores.
English multiple choice via inspect_ai.solver.multiple_choice. BFI: five options A–E (Disagree strongly … Agree Strongly), 16 reverse-keyed items. TRAIT: four options (two high-trait, two low-trait responses). Optional -T personality=… system prompt. TRAIT shuffle in {questions, choices, all} with a seed. Output format ANSWER: $LETTER.
No model card in ModelSpec reports this benchmark yet.