IFEval-style verifiable instruction following extended to three-turn conversations in eight languages, 4,501 dialogues.
unassessed
| Category | instruction-following |
|---|---|
| Subcategory | multi-turn multilingual verifiable instruction following |
| Page status | active |
| Metric | mean of prompt-level and instruction-level strict and loose accuracy, per turn and overall |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 4501 |
| Dataset licence | CC BY-NC 2.0 |
| Publisher | Meta (Facebook AI) |
Multi-IF tests whether a model still obeys mechanically checkable instructions when those constraints span three dialogue turns and eight languages. Each user turn adds or restates IFEval-style rules (length, keywords, format). Scoring uses the same strict and loose program checks as [IFEval](ifeval.md), applied per turn. The English prompts were translated into seven other languages with an LLM, then audited by professional annotators.
Three-turn chat. The model answers turn 1, then turn 2 given the history, then turn 3. Each turn has an instruction-id list and kwargs for the IFEval checkers. OpenCompass MultiIFDataset builds a six-message dialogue (user/assistant × 3) and scores with MultiIFEvaluator. Official code is facebookresearch/Multi-IF on the `facebook/Multi-IF` CSV.
No model card in ModelSpec reports this benchmark yet.