Cyber Risks · Trail of Bits Blog

Prompt injection engineering for attackers: Exploiting GitHub Copilot

Summary: Trail of Bits designed a prompt injection exploit that hides instructions in an HTML <picture> tag inside a GitHub issue, invisible to maintainers but readable by GitHub Copilot, so that Copilot inserts a backdoor into a project's uv.lock file when assigned the issue. The backdoored dependency executes bash commands sent in the X-Backdoor-Cmd HTTP header, and the demonstration targeted a Flask web application in the trailofbits/copilot-prompt-injection-demo repository.

Key Takeaway: The attack requires only that a maintainer assign an issue to Copilot, and the resulting pull request looks like an ordinary dependency change that reviewers rarely inspect line by line. Any open-source project that lets Copilot act on externally filed issues is exposed to a backdoor that ships with the next deployed release.

Read the original at Trail of Bits Blog ↗

Topics
AI Security
Related