A Hindi BIG-bench yes/no task that asks what polarity an implicative verb entails, translated from entailed_polarity.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | BIG-bench Hindi yes/no implicative-verb polarity (142 JSON items) |
| Page status | unknown |
| Metric | multiple_choice_grade |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 142 |
| Dataset licence | Apache-2.0 |
| Publisher | Google (BIG-bench collaboration) |
entailed_polarity_hindi is the Hindi counterpart of [entailed_polarity](entailed_polarity.md). Each item is a Hindi fact that uses an implicative construction, then a yes/no question (हां / नहीं) about the entailed event. Kaustubh Dhole translated the English pairs with Google Translate and edited them by hand. The README says the set covers the Hindi equivalent of all English verbs and examples, but English–Hindi mappings are not one-to-one (some English frames lack a Hindi counterpart; some Hindi sentences have several phrasings). The skill is implicative polarity in Hindi, not English NLI and not a general Hindi exam.
Two-option multiple choice (हां / नहीं), preferred metric multiple_choice_grade. Hindi task_prefix asks to answer with yes or no. append_choices_to_input is false. Canary GUID embedded.
No model card in ModelSpec reports this benchmark yet.