Microsoft's adversarial-prompt robustness eval: character- to semantic-level attacks on the instruction, scored as the drop on GLUE, MMLU, SQuAD, translation and math.
unassessed
| Category | safety |
|---|---|
| Subcategory | adversarial prompt robustness on existing NLU, QA, translation and math tasks |
| Page status | active |
| Metric | task metric under attack, often reported as performance drop versus the clean prompt |
| Direction | higher_is_better |
| Dataset licence | MIT (PromptBench/PromptRobust code repository, Microsoft Corporation) |
| Publisher | Microsoft Research, with Institute of Automation CAS, Carnegie Mellon University, Peking University, Westlake University and Duke University |
PromptBench, later retitled PromptRobust on arXiv, measures whether a model still solves an ordinary task after the instruction — not the passage — is perturbed. Attacks run at four levels: character (typos), word (synonym substitution), sentence (appended distractors) and semantic (cross-country English). The same adversarial instructions are applied across eight task types and thirteen datasets, including GLUE-style classification, MMLU, SQuAD v2, UN Multi, IWSLT 2017 and a mathematics set. OpenCompass ships four of those datasets under configs/datasets/promptbench, using AttackInferencer.
Task-dependent generation or classification, but with the instruction slot filled by an attacked prompt (`adv_prompt`) drawn from a list of clean paraphrases. OpenCompass covers IWSLT 2017 English-German (BLEU, BM25 1-shot), MATH (generation), SQuAD 2.0 (span/unanswerable) and WNLI (A/B entailment). The original paper evaluates a broader 13-dataset mix.
No model card in ModelSpec reports this benchmark yet.