SWE-Gym is a training environment of real Python issue-resolution tasks; agents trained on it are typically reported by resolve rate on SWE-bench Verified and Lite, not a separate SWE-Gym leaderboard.
unassessed
| Category | coding |
|---|---|
| Subcategory | agent training environment with held-out evaluation |
| Page status | active |
| Metric | % resolved (patch passes FAIL_TO_PASS and PASS_TO_PASS tests) |
| Direction | higher_is_better |
| Unit | % |
| Dataset size | 2438 |
| Dataset licence | Apache-2.0 |
SWE-Gym provides real-world Python task instances (a codebase snapshot, an executable runtime, unit tests and a natural-language issue) for training and verifying software-engineering agents. It is built to be run against, not just read: an agent explores the repository, edits code and executes tests inside the provided environment. The paper's own reported numbers measure what training on SWE-Gym does for resolve rate on the separate SWE-bench Verified and Lite test sets, plus a "Lite" 234-instance subset of SWE-Gym itself used for lighter-weight evaluation during development.
Text input (issue description) with repository and executable test access; output is a patch, graded by running hidden tests recovered from the originating pull request.
No model card in ModelSpec reports this benchmark yet.