Dynamic analysis runs the software on specific inputs and checks results against an oracle. It cannot test all inputs: adding 2 64-bit integers has cases, so exhaustive testing is infeasible. Security requirements are often of the form “X never happens”, which testing cannot prove.
Negative Testing
Negative testing checks what should not happen, not just what should.
Examples:
- Can I read without authorization?
- Can I connect with an invalid certificate?
Web Application Scanners
Web application scanners crawl forms and links like a browser, send attack-like and random data, and detect problems.
Examples:
- OWASP ZAP
- W3AF
- Burp Suite Pro
Fuzz Testing
Fuzz testing feeds large numbers of random or mutated inputs. It monitors for crashes, assertion failures, and memory leaks, not for correct answers. The concept comes from Barton Miller, 1988. It often finds real defects quickly, with diminishing returns after fixes.
Input strategies:
- Fully random
- Mutation-based
Aka. dumb fuzzing, mutates sample inputs. - Generation-based
Aka. smart fuzzing, models the input format.
Coverage-guided evolution routes inputs that hit new paths for more mutation.
American Fuzzy Lop (AFL, AFL++) is a coverage-guided mutation fuzzer. It instruments branch counts and prioritizes inputs hitting new edges. It once produced a valid JPEG starting from the text “hello”.
Address Sanitizer
Address Sanitizer (ASan) compiles with -fsanitize=address to catch buffer overflow, use-after-free, and double-free. It uses shadow bytes and inaccessible red zones around allocations.
Overhead is roughly 2x CPU and 2x to 4x memory. Sanitizers tolerate some false positives. Production exploit mitigations tolerate none.
