Crawling volatile content
Summary: PortSwigger describes how its new Burp crawler handles volatile content, where the same location returns substantially different responses on different visits due to social media feeds, user comments, inline advertising, or randomized content such as A/B testing. Unlike the current Spider, which fetches each request once, the new crawler navigates only via intended pathways, identifies locations by their contents, and when responses differ too much to be treated as the same, captures both versions as separate locations and plots a non-deterministic edge in the graph.
What you should know: If the extent of non-determinism across an application is not too great, Burp can still crawl the associated content and reliably reach content behind a non-deterministic link - which means the coverage of your scan depends on how much of your application changes between visits.
Read the original at PortSwigger Research ↗
- Topics
- Vulnerabilities & Patching
- Related
- Attackers Exploit UnrealIRCd Flaw That Enables Server-Level Command Injection Cyber Risks · Nov 14, 2026U.S. CISA adds Check Point and UnrealIRCd flaws to its Known Exploited Vulnerabilities catalog Cyber Risks · Nov 14, 2026PostgreSQL Patches 52 Flaws, Including 22-Year-Old Bug Enabling Authenticated RCE Cyber Risks · Nov 13, 2026