Thursday, March 12, 2009

ETech: Bug Finding by Solving Constraints in the Cloud

The last day of most O'Reilly conferences always seem a lot quieter. Possibly because they run at such a pace that most people are physically exhausted by the end of them. In any case, both OSCON and ETech tend to have a slower feel to them on the final day. This ETech doesn't seem to be the exception to the general rule, there are fewer people about, and many fewer people in the talks...

I'm kicking off my day with "Bug Finding by Solving Constraints in the Cloud" given by David Molnar.

Security bugs are costly, and there is a bug cycle...

Classic technique fro finding bugs is Fuzz Testing where you feed malformed files, strings and commands to your application and see whether it crashes. Interestingly, despite this being a fairly basic technique, it finds lots of bugs. But fuzz testing doesn't work well with unlikely paths, the edge cases. The fix being talking about today is fuzz testing plus dynamic test generation.

Trace the dynamic execution of the program, capture the symbolic path conditions, create symbolic formula for new path, solve new oath conditions and generate a new test case from solution.

Does this scale? The first generation of tools, like EGT, were exciting but only tested cases where the program was less than 2KLOC of code.

He's now talking about two tools. SAGE, which he worked on at Microsoft and is still internal, and SmartFuzz. You can actually get SmartFuzz, and it scales to larger code by running it in the cloud on Amazon EC2 instances.