HELM's Vietnamese–English translation pair of OPUS-100 and PhoMT, scored mainly by quasi-exact match rather than BLEU.
unassessed
| Category | translation |
|---|---|
| Subcategory | HELM Vietnamese–English sentence translation (OPUS-100 and PhoMT) |
| Page status | unknown |
| Metric | quasi_exact_match (headline); exact_match, f1_score, rouge_l, bleu_1, bleu_4 also recorded |
| Direction | higher_is_better |
| Unit | % |
| Publisher | Stanford CRFM (HELM); OPUS-100 from Zhang, Williams, Titov and Sennrich; PhoMT from VinAI (Doan, Nguyen, Tran, Hoang, Nguyen) |
melt_translation is HELM's Vietnamese–English sentence translation wrapper. A run gives the model a source sentence in English or Vietnamese and asks it to produce the other language. The parent Scenario class only accepts the pair (vi, en) with exactly one side English. Two concrete datasets are wired: OPUS-100 English–Vietnamese (Hugging Face vietgpt/opus100_envi) and PhoMT (Hugging Face ura-hcmut/PhoMT, a copy of VinAI's PhoMT). HELM's schema groups both under "MELT Scenarios", a Vietnamese evaluation suite contributed to HELM; this page does not treat the schema's leftover "medical domain" blurb as a description of these tasks. It is text-only machine translation, not FLORES and not a general multilingual MT leaderboard.
Generate a translation of the source sentence into the target language. The adapter instruction is "Translate the following sentences from {source} to {target}." with lines labelled "English:" / "Vietnamese:". Default in-context examples: 1. Language pair is a run argument (en-vi or vi-en).
No model card in ModelSpec reports this benchmark yet.