A 1,003-example BIG-bench task that transliterates sentences between English and the International Phonetic Alphabet.
unassessed
| Category | translation |
|---|---|
| Subcategory | BIG-bench sentence-level English–IPA transliteration |
| Page status | unknown |
| Metric | bleu |
| Direction | higher_is_better |
| Dataset size | 1003 |
| Dataset licence | OANC End User License |
| Publisher | Google (BIG-bench collaboration) |
international_phonetic_alphabet_transliterate asks a model to convert a sentence from English orthography into IPA, or the reverse. The authors treat it as a probe of rare-symbol handling and of whether pretraining stored word-level pronunciation maps, not as a speech benchmark. Sentences are drawn from MultiNLI generated (non-web) hypotheses under the OANC licence, fiction excluded, then converted with the CMU Pronouncing Dictionary via eng-to-ipa. The prompt prefix gives three mixed-direction exemplars.
Free-text transliteration. task.json preferred_score is bleu; metrics also include rouge and exact_str_match. Keywords include many-shot. Canary GUID embedded. Direction is encoded in the input (IPA: … versus English: … plus an IPA: suffix).
No model card in ModelSpec reports this benchmark yet.