Cyber Risks · Trail of Bits Blog

Datasig: Fingerprinting AI/ML datasets to stop data-borne attacks

Summary: Trail of Bits released Datasig, a lightweight tool that generates compact MinHash-based fingerprints of AI/ML datasets so training data can be compared with high accuracy without access to the raw data. In experiments on the MNIST vision dataset, fingerprints of 400 hashes produced a bounded error margin as small as 5%, with longer fingerprints trading heavier computation for greater accuracy.

Bottom line: Attackers can poison training data to backdoor models, leak private information, or introduce bias, and most organizations cannot answer what data actually trained a given model - so without dataset traceability, integrity checks, compliance audits, and incident investigations have no foundation.

Read the original at Trail of Bits Blog ↗

Topics
AI Security
Related