Cyber Risks · Trail of Bits Blog

Leftover Locals: Listening to LLM responses through leaked GPU local memory

Summary: Trail of Bits disclosed LeftoverLocals (CVE-2023-4969), a vulnerability that lets one process recover data from GPU local memory written by another process on Apple, Qualcomm, AMD, and Imagination GPUs. The researchers built a proof of concept that listens in on another user's interactive LLM session across process or container boundaries, leaking about 5.5 MB per GPU invocation on an AMD Radeon RX 7900 XT, or roughly 181 MB per query when running a 7B model on llama.cpp.

What's compelling: An attacker needs only the ability to run GPU compute applications through OpenCL, Vulkan, or Metal, and the exploit can be under 10 lines of code, so any co-resident user or container on a shared GPU host is a potential listener. With roughly 181 MB leaked per LLM query, enough to reconstruct the response with high precision, multi-tenant GPU inference carries a data-exposure path that current tooling does not log or check.

Read the original at Trail of Bits Blog ↗

Topics
Vulnerabilities & Patching
Mentions
CVE-2023-4969
Related