1,976 Bangla yes/no reading-comprehension questions GPT-4-generated from Bangla Wikipedia, Banglapedia and news passages; inspired by BoolQ's format but not a translation of it.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | Bangla yes/no reading comprehension, GPT-4-generated rather than translated |
| Page status | active |
| Metric | accuracy |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 1976 |
| Dataset licence | MIT (Hugging Face dataset card) |
| Publisher | Hishab (Hishab Singapore Pte. Ltd), with the University of Central Florida and the Qatar Computing Research Institute |
Bangla BoolQA tests whether a model can answer a yes/no question about a short Bangla passage, following the same task shape as the English BoolQ: a (passage, question, answer) triplet where the answer is "yes" or "no". It differs from the other three Bangla benchmarks introduced alongside it (bangla_commonsenseqa, bangla_piqa, bangla_openbookqa) in one important way: it is not a translation of the English dataset. The TituLLMs paper that introduces it describes it as "inspired by BoolQ", built from scratch with passages sourced from Bangla Wikipedia, Banglapedia and Bangla news articles, and yes/no questions generated over those passages by GPT-4. That construction method changes how a score should be read: unlike a translation, where the exact same 15,942 English items reappear in Bangla, this dataset is a fresh, natively-sourced set of 1,976 items that only shares BoolQ's task format and inspiration, not its item pool.
Binary yes/no question answering given a short Bangla passage. The reference lm-evaluation-harness task scores it zero- or few-shot as a multiple-choice comparison between the log-likelihood the model assigns to the Bangla words for "yes" and "no", rather than free-form generation.
No model card in ModelSpec reports this benchmark yet.