lm-eval tag/group for Hendrycks ETHICS: five English tasks of everyday moral judgment, scored as per-item accuracy on the public test split.
unassessed
| Category | safety |
|---|---|
| Subcategory | five-task English moral-judgment suite (justice, deontology, virtue, utilitarianism, commonsense) |
| Page status | unknown |
| Metric | accuracy (acc) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 19968 |
| Dataset licence | other |
| Publisher | UC Berkeley and collaborators (dataset); EleutherAI (lm-eval tasks) |
hendrycks_ethics is EleutherAI lm-evaluation-harness's tag over the ETHICS dataset from Hendrycks et al. The model reads a short English scenario and predicts a widespread moral judgment: wrong versus not wrong (commonsense), reasonable versus unreasonable (justice, deontology), whether a trait is shown (virtue), or which of two scenes is preferable (utilitarianism). The paper's point is connecting physical and social knowledge to value judgments, not solving dilemmas. This id is the harness wrap of the public test CSVs. It is not [decodingtrust_machine_ethics](decodingtrust_machine_ethics.md) (a HELM wrap that also uses jailbreaks and Jiminy Cricket) and not [hhh_alignment](hhh_alignment.md).
Multiple choice, test split only. Commonsense prompt: "{input}\nQuestion: Is this wrong?\nAnswer:" with choices no/yes. Justice and deontology ask whether a quoted claim is reasonable. Virtue asks whether a named trait is exhibited. Utilitarianism shuffles activity versus baseline with a per-item RNG seed and asks if Scenario 1 is preferable. training_split is declared but scoring uses test. Hard Test CSVs exist in the authors' tarball; the EleutherAI loader comments that the hard splits are not implemented and does not emit them.
No model card in ModelSpec reports this benchmark yet.