A 541-prompt professional Spanish translation of IFEval, checked by a separately reimplemented Spanish instruction-verification codebase rather than IFEval's English checkers.
unassessed
| Category | instruction-following |
|---|---|
| Subcategory | Spanish translation of IFEval's verifiable-instruction suite |
| Page status | active |
| Metric | instruction-following accuracy (strict and loose, at the prompt level and the instruction level) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 541 |
| Dataset licence | CC BY 4.0 |
| Publisher | Barcelona Supercomputing Center (BSC-CNS), Language Technologies Unit |
ifeval_es is the Barcelona Supercomputing Center's professional Spanish translation of Google's IFEval: 541 prompts, matching the 541-row English source ("train" split of google/IFEval) one for one. As with the sibling Catalan translation, lm-evaluation-harness does not run the translated prompts through IFEval's English checking code; it ships a dedicated Spanish instruction registry that reimplements every checker, using `langdetect` language detection and Unicode-aware case folding so Spanish text is judged by Spanish-appropriate rules. The registry defines 30 Spanish instruction types in code -- the original 25 English types (the two Latin-alphabet capitalisation checks become Spanish-alphabet versions) plus 5 new ones (two extra punctuation checks, three "special character" checks for the letter n-with-tilde and accented vowels) -- but the 541 released prompts only ever invoke the original 25, confirmed by reading every prompt's instruction_id_list directly. Unlike the Catalan sibling file, where the equivalent unused checks test for the wrong language's diacritics, the Spanish special-character checks do target genuine Spanish orthography; they are simply dormant, exercised by no released prompt.
A model is given a Spanish prompt containing one or more verifiable instructions and generates a free-form Spanish response in a single turn, zero-shot. lm-evaluation-harness's `ifeval_es` task generates greedily (do_sample: false) with a 1,280-token cap, then a Spanish-specific verification function checks the response against each instruction mechanically, mirroring the English `ifeval` task's process_results structure.
No model card in ModelSpec reports this benchmark yet.