How we use /goal to find bugs in Patch the Planet

Summary: Trail of Bits engineers, working through Patch the Planet with OpenAI, pointed Codex's /goal feature at heavily audited codebases including Rust, curl, and zlib, and converged on three techniques for goal-based prompting. One engineer, Kevin Valerio, built an automated variant-analysis pipeline for the Rust compiler that produced every Rust bug the team submitted, using one Codex session per P-critical issue with a two-model false-positive check and a final human filter.
Key Takeaway: Rust maintainers assumed a team of engineers was hunting bugs when it was one engineer with /goal, and a single-sentence bug-class prompt surfaced numerous variants of which 9 were reported and 3 already fixed upstream - the same amplification is available to anyone pointing these tools at your code.

Read the original at Trail of Bits Blog ↗