A BIG-bench task that asks a model to classify a comment from an online support community as supportive, neutral or unsupportive of the post it replies to.
unassessed
| Category | safety |
|---|---|
| Subcategory | social and emotional understanding: classify supportiveness of a comment in an online conversation |
| Page status | active |
| Metric | Macro-F1 against majority-vote crowd labels (BIG-bench also records multiple_choice_grade) |
| Direction | higher_is_better |
| Unit | F1 |
| Dataset size | 897 |
| Publisher | Google (BIG-bench collaboration); task authors Zijian Wang and David Jurgens |
Social Support gives a model a post from an online community and a reply comment, and asks the model to judge whether the reply is supportive, neutral or unsupportive of the poster. The annotated corpus behind the task spans Reddit, StackExchange and Wikipedia talk-page interactions, not a single platform. It targets a narrow slice of social and emotional understanding: recognising encouragement, empathy or advice versus dismissiveness or hostility in short, informal, emotionally loaded text, not general sentiment polarity.
Zero-shot multiple-choice classification. Each of 897 examples presents a post/reply pair and asks the model to pick one of three labels (supportive, neutral, unsupportive); BIG-bench scores it via the multiple_choice_grade metric over the three answer options.
No model card in ModelSpec reports this benchmark yet.