A BIG-bench wrap of MultiEmo: four-way sentiment on consumer reviews in 11 languages at document and sentence level.
unassessed
| Category | knowledge |
|---|---|
| Subcategory | BIG-bench four-way multilingual review sentiment (document and sentence) |
| Page status | unknown |
| Metric | multiple_choice_grade |
| Direction | higher_is_better |
| Unit | % |
| Dataset licence | CC-BY-4.0 (MultiEmo corpus, per the task README); Apache-2.0 (BIG-bench task packaging) |
| Publisher | Wrocław University of Science and Technology / CLARIN-PL (BIG-bench collaboration) |
multiemo asks whether a consumer review, or one of its sentences, is positive, negative, neutral, or ambivalent. The underlying corpus is Polish reviews from hotels, medicine, products and universities, annotated by three linguists per item, then machine-translated into ten other languages. BIG-bench ships 110 JSON subtasks: five groupings (all, hotels, medicine, products, reviews) × sentence or whole-text × 11 languages. English text in the en files is a translation of the Polish source, not native English reviews. It is not [emojis_emotion_prediction](emojis_emotion_prediction.md).
Four-option multiple choice with preferred metric multiple_choice_grade. task_prefix asks the model to label the opinion positive, negative, neutral or ambivalent. Canary GUID embedded. Zero-shot in the task keywords.
No model card in ModelSpec reports this benchmark yet.