Cyber Risks · Trail of Bits Blog

Exploiting ML models with pickle file attacks: Part 1

Summary: Trail of Bits developed a technique called Sleepy Pickle that injects a malicious payload into the pickle file packaging an ML model, so that deserialization modifies the model in place rather than attacking the host system. The researchers demonstrated proof-of-concept attacks that patched GPT-2-XL to advise that drinking bleach cures the flu, hooked a summarizer model's inference function to record private user data until a trigger word returned it, and inserted malicious links into generated summaries.

What's compelling: The model-weight patch that poisoned GPT-2-XL added less than 0.1% overhead to the pickle file, and the data-theft variant runs entirely inside the model code through the application's public interface, so DLP and firewalls never see it.

Read the original at Trail of Bits Blog ↗