Forest Global Training Partners

Cyber Risks · Unit 42 Palo Alto Networks

The Risks of Code Assistant LLMs: Harmful Content, Misuse and Deception

Summary: Unit 42 researchers found that AI code assistants connected to IDEs as plugins can be misused by users and threat actors to inject backdoors, leak sensitive information and generate harmful content, and that context attachment features are vulnerable to indirect prompt injection through contaminated public or third-party data sources. In a simulated scenario, a tainted sample of X posts fed to an assistant as context caused it to insert a hidden backdoor called fetched_additional_data that would fetch a remote command from an attacker-controlled command-and-control server and execute it.

Bottom line: The 2024 Stack Overflow Annual Developer Survey cited in the research found 76% of respondents are using or plan to use AI tools in development, and 82% of current users employ them to write code - so a contaminated context source can turn a routine copy-and-paste into a compromised developer machine.

Read the original at Unit 42 Palo Alto Networks ↗

Topics
AI Security
Related