200 German news sentences rewritten into gender-inclusive forms; BIG-bench scores exact string match against author targets.
unassessed
| Category | generation |
|---|---|
| Subcategory | German rewrite into gender-inclusive morphology (BIG-bench) |
| Page status | unknown |
| Metric | exact_str_match |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 200 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
The model reads a German sentence that uses masculine generics or other non-inclusive person reference and must rewrite it with gender-inclusive morphology. The authors' preferred device is the star form (Regisseur*in, der*die) that names masculine and feminine forms and leaves room for non-binary reference; gender-neutral words are allowed when the star form is unused or unnatural. The skill is spotting which nouns are abstract persons rather than named individuals, then updating articles, adjectives, and relative pronouns. Inputs are German; this is not the English or Chinese occupation-bias probes in gender_sensitivity_*.
Free-response. Prefixes: "Non-gender-inclusive:" / "Gender inclusive:". Preferred metric exact_str_match. 135 items have one string target; 65 have a list of two to seven accepted rewrites (53 with two). Canary GUID embedded. Dummy-model header: 0 multiple-choice and 200 free-text queries.
No model card in ModelSpec reports this benchmark yet.