3,400 human-annotated math solutions, mostly contest-level, where the model must name the earliest wrong step or report that the trace is clean.
unassessed
| Category | math |
|---|---|
| Subcategory | step-level error localisation in mathematical reasoning traces |
| Page status | active |
| Metric | F1 of error_acc and correct_acc |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 3400 |
| Dataset licence | Apache-2.0 |
| Publisher | Qwen Team, Alibaba Inc. |
ProcessBench tests whether a model can find mistakes in someone else's math, not whether it can solve the problem from scratch. Each item is a contest- or school-level problem plus a step-by-step solution whose first error (or lack of error) was labelled by human experts. The model must return the index of that earliest bad step, or -1 if every step is correct. The suite is aimed at process reward models and at general models prompted as critics.
English generation. OpenCompass wraps each step in <paragraph_i> tags and asks for a paragraph-by-paragraph critique with the final index in \\boxed{}. Official Qwen code uses the same earliest-error / -1 protocol. Subsets: gsm8k, math, olympiadbench, omnimath.
No model card in ModelSpec reports this benchmark yet.