Our thoughts on AIxCC’s competition format
Summary: DARPA opened registration for its AI Cyber Challenge and released a request for comments containing a sample challenge problem and the scoring methodology. Competing teams must build fully automated cyber reasoning systems that find and patch vulnerabilities in modified real-world software such as the Linux kernel and Jenkins, submitting a proof of vulnerability, a proof of understanding, and optionally a patch, with at least half the challenge problems being C/C++ programs containing memory corruption vulnerabilities.
Bottom line: Trail of Bits warns that challenge problems built by reinjecting previously disclosed vulnerabilities are almost certainly in the training data of commercial LLMs like ChatGPT and Claude, so LLM-based systems that score highly in AIxCC will likely struggle when used outside the competition.
Read the original at Trail of Bits Blog ↗
- Topics
- Vulnerabilities & Patching
- Related
- Attackers Exploit UnrealIRCd Flaw That Enables Server-Level Command Injection Cyber Risks · Nov 14, 2026U.S. CISA adds Check Point and UnrealIRCd flaws to its Known Exploited Vulnerabilities catalog Cyber Risks · Nov 14, 2026PostgreSQL Patches 52 Flaws, Including 22-Year-Old Bug Enabling Authenticated RCE Cyber Risks · Nov 13, 2026