Cyber Risks · Trail of Bits Blog

Try our new dimensional analysis Claude plugin

Summary: Trail of Bits released a Claude Code plugin that uses an LLM to annotate a codebase with dimensional types and then flags mismatches mechanically, rather than asking the model to find bugs directly. In testing against dimensional mismatch issues from several unpublished audits, the plugin achieved 93% recall with a standard deviation of 12%, versus 50% recall with a standard deviation of 20% for a baseline prompt.

What you should know: The plugin's advantage comes from not trusting LLM judgment to rank findings: the model builds a vocabulary and annotates, and a mismatch between annotations surfaces the bug, which is why recall held at 93% against 50% for baseline prompts. Trail of Bits states the tool cannot find every source of dimensional error, since LLMs are probabilistic.

Read the original at Trail of Bits Blog ↗

Topics
Vulnerabilities & Patching
Related