3CB (Catastrophic Cyber Capabilities Benchmark) scores LLM agents on capture-the-flag cyber offense challenges mapped to MITRE ATT&CK techniques.
unassessed
| Category | agentic |
|---|---|
| Subcategory | cyber offense capture-the-flag |
| Page status | active |
| Metric | success rate |
| Direction | higher_is_better |
| Unit | fraction |
| Dataset size | 15 |
| Dataset licence | CC BY-NC-SA 4.0 (paper/dataset, per arXiv) |
| Publisher | Apart Research |
3CB tests whether an LLM agent can perform offensive cyber tasks - reconnaissance, exploitation, privilege escalation, and related techniques - by solving capture-the-flag (CTF) challenges in a bash shell, each mapped to a MITRE ATT&CK technique.
Agent interacts with a containerized challenge environment (typically a bash shell) and must recover a flag string that proves successful exploitation.
No model card in ModelSpec reports this benchmark yet.