ToxiGen

A machine-generated dataset of 274,000 toxic and benign statements about 13 minority groups, used to test whether models can catch subtle, implicit hate speech.

unassessed

What it measures

ToxiGen targets implicit and adversarial toxicity - text that is hateful in effect without using slurs or other easily-flagged surface markers - directed at 13 minority groups. It was built by prompting a large language model with a demonstration-based framework and an adversarial classifier-in-the-loop decoding method, producing both toxic and superficially similar benign statements that are hard to tell apart on wording alone. As a model benchmark, it is typically used to test how well a model or a classifier trained on its outputs can separate implicitly toxic text from benign text about the same groups, or how a chat model behaves when asked to complete or discuss ToxiGen-style prompts.

Task format

Binary classification of statements as toxic or benign (as a classifier benchmark), or completion/discussion of ToxiGen-style prompts scored for toxic output (as a generation safety check).

Models reporting this benchmark

Data

This page as JSON · Edit on GitHub