16 Expanding Fuzzing Of Critical Program Configurations Via Coverage-Based Differential Testing

Faculty Mentor: Stefan Nagy (Kahlert School of Computing, University of Utah)

Modern software is configurable, meaning it has features that can be enabled or disabled to affect the program’s output and behavior. However, configurable code can be difficult to test as each configuration (i.e., each combination of features) can introduce new execution paths, and therefore new bugs. Ideally, each configuration would be tested, but in practice, this is infeasible, as many software products have prohibitively many configurations to test in-depth. This problem can be mitigated by selecting and testing common configurations, but this is insufficient to safeguard the many variant configurations that will be used in practice.

In my work, I develop a new, semi-automated tool to reduce the difficulty of testing configurable software. The tool generates compile-time configurations and tests them against a fixed set of inputs, performing a kind of configuration-fuzzing as opposed to traditional input-fuzzing. The tool identifies critical compile-time configurations that exercise code not reached by the default configuration. These “interesting” configurations can later be tested in-depth by traditional fuzzers to reveal bugs not found in the default configuration. This coverage-driven, differential exploration of the compile-time configuration space is the main innovation of my approach.

I evaluate this tool by testing open-source libraries with publicly available fuzzing harnesses (primarily libraries hosted by Google’s OSS-Fuzz). With this work, I hope to provide a way of efficiently testing programs with large compile-time configuration spaces.