Polish four-class review sentiment from PolEmo 2.0, scored in-domain and out-of-domain as two lm-evaluation-harness tasks.
unassessed
| Category | domain |
|---|---|
| Subcategory | four-class Polish consumer-review sentiment, in-domain and out-of-domain |
| Page status | unknown |
| Metric | micro-F1 and accuracy (Hugging Face evaluate); KLEJ cards also quote accuracy |
| Direction | higher_is_better |
| Dataset size | 1216 |
| Dataset licence | CC-BY-NC-SA-4.0 |
| Publisher | Wrocław University of Science and Technology and University of Wrocław; KLEJ packaging by Allegro |
PolEmo 2.0 is a Polish consumer-review corpus. The model reads a review and must label it neutral, negative, positive, or ambiguous. KLEJ splits the same hotel and medicine training reviews into an in-domain test (hotels and medicine) and an out-of-domain test (products and university). lm-evaluation-harness follows that pair as polemo2_in and polemo2_out. It is review-level four-way sentiment, not sentence-level tagging, and not English IMDb-style polarity.
lm-eval generate_until. Polish prompt "Opinia:" plus four lettered options (A Neutralny, B Negatywny, C Pozytywny, D Niejednoznaczny). Decoding stops at "." or ",", temperature 0, max 50 tokens. A regex keeps the first A–D letter. Gold labels are __label__meta_zero, __label__meta_minus_m, __label__meta_plus_m, __label__meta_amb. Tag polemo2 runs both tasks.
No model card in ModelSpec reports this benchmark yet.