10,962-item machine translation of CommonsenseQA into Bangla, built with an automated Google-Translate-plus-LLM-rewriting pipeline the authors call Expressive Semantic Translation.
unassessed
| Category | reasoning |
|---|---|
| Subcategory | Bangla commonsense multiple-choice QA, machine-translated from CommonsenseQA |
| Page status | active |
| Metric | accuracy (acc and length-normalised acc_norm) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 10962 |
| Dataset licence | MIT (Hugging Face dataset card) |
| Publisher | Hishab (Hishab Singapore Pte. Ltd), with the University of Central Florida and the Qatar Computing Research Institute |
Bangla CommonsenseQA tests the same thing as the English CommonsenseQA -- whether a model can answer a five-way multiple-choice question that requires general world knowledge rather than information stated in the question itself -- but in Bangla. It is a translation, not a from-scratch Bangla benchmark: the TituLLMs paper that introduces it states plainly that the authors "translated the CommonsenseQA dataset ... into Bangla" using a custom pipeline they call Expressive Semantic Translation (EST), which combines a standard neural machine translation pass with an iterative, LLM-based refinement step that generates and ranks multiple candidate re-translations before selecting one. No human translation or per-item human verification of the translated benchmark items themselves is described for this dataset, which distinguishes it from `bangla_boolqa` in the same paper, where a human annotator did spot-check the (independently generated, not translated) items.
Five-way multiple-choice question answering in Bangla (answer labels A-E), no supporting passage; the reference lm-evaluation-harness task scores it zero- or few-shot by comparing the log-likelihood the model assigns to each of the five answer options.
No model card in ModelSpec reports this benchmark yet.