Multi-task Hindi and Odia dialect suite: 11-way classification, translation MCQ, and dialect-standard MT over 13,000 native-speaker sentence pairs.
unassessed
| Category | composite |
|---|---|
| Subcategory | Hindi and Odia dialect classification, translation MCQ, and dialect-standard machine translation |
| Page status | unknown |
| Metric | task-specific: classification and MCQ F1; MT BLEU (Papineni et al. 2002) |
| Direction | higher_is_better |
| Dataset size | 13000 |
| Publisher | Indian Institute of Technology Mandi; Indian Institute of Technology Kanpur |
INDIC-DIALECT tests whether a model can handle regional Hindi and Odia dialects rather than only the standardized written forms used in news and Wikipedia. The suite has three tasks on the same parallel corpus. Dialect classification asks which of 11 named dialects a sentence belongs to. An MCQ task shows one dialect sentence and four standard-language candidates, one of them the true translation and three hard near-miss distractors. Machine translation runs in both directions: dialect to standard Hindi or Odia, and standard language to dialect. The skill is dialect-aware Indic NLP, not scheduled-language NLI or FLORES-style standard MT.
Text in. Classification emits a dialect name. MCQ picks one of four standard-language options. MT emits a free-text translation. The paper's LLM classification prompt asks the model to reply with only the dialect name.
No model card in ModelSpec reports this benchmark yet.