Cyber Risks · The Hacker News

Microsoft Develops Scanner to Detect Backdoors in Open-Weight Large Language Models

Summary: Microsoft's AI Security team said on Wednesday it built a lightweight scanner that detects backdoors in open-weight large language models by relying on three observable signals, with researchers Blake Bullwinkel and Giorgio Severi citing memorized poisoning data and distinctive patterns in output distributions and attention heads when triggers are present. The scanner extracts memorized content, isolates salient substrings, and scores them as loss functions to return a ranked list of trigger candidates, requiring no additional training or prior knowledge of the backdoor behavior, though it does not work on proprietary models and works best on trigger-based backdoors with deterministic outputs.

Bottom line: A poisoned open-weight model behaves normally until a narrow trigger fires, so the exposure sits in every model file your teams download and fine-tune, and Microsoft's own tool cannot inspect the proprietary models you cannot access.

Read the original at The Hacker News ↗

Topics
AI Security
Related