Cyber Risks · Trail of Bits Blog

Our thoughts on AIxCC’s competition format

Summary: DARPA opened registration for its AI Cyber Challenge and released a request for comments containing a sample challenge problem and the scoring methodology. Competing teams must build fully automated cyber reasoning systems that find and patch vulnerabilities in modified real-world software such as the Linux kernel and Jenkins, submitting a proof of vulnerability, a proof of understanding, and optionally a patch, with at least half the challenge problems being C/C++ programs containing memory corruption vulnerabilities.

Bottom line: Trail of Bits warns that challenge problems built by reinjecting previously disclosed vulnerabilities are almost certainly in the training data of commercial LLMs like ChatGPT and Claude, so LLM-based systems that score highly in AIxCC will likely struggle when used outside the competition.

Read the original at Trail of Bits Blog ↗

Topics
Vulnerabilities & Patching
Related