Your AI agent fixed the bug, but what else did it change?

Summary: Trail of Bits released two plugins, patching-bugs and regression-hunt, that delegate patch development and regression checks to AI agents, built through its Patch the Planet initiative with OpenAI. A review of every merged Patch the Planet patch found 21 regressions across more than 130 merged PRs - 16 functional, one performance, and four build/test - none of which broke a security protection.
Bottom line: The 21 confirmed regressions came from 58 candidates in a single run, about 36% precision, and regression-hunt cut false positives 25% versus a baseline Codex agent while leaving some known regressions classified as borderline for lack of evidence. An agent that ships a fix can also ship a defect a real user or attacker can reach, and the review process that catches it is still noisy enough to miss some.

Read the original at Trail of Bits Blog ↗