Please note: This master’s research paper presentation will take place in DC 2314.
Rui Ming Xiong, Master’s candidate
David R. Cheriton School of Computer Science
Supervisors: Professors Joanne Atlee, Chengnian Sun
Compiler testing approaches generally either generate fresh programs or mutate existing programs. This paper proposes two mutation-based methods, mutating the literal values found in a seed program to create compiler test cases. By turning the literal values into variables, and using the symbolic execution engine KLEE, we find new values for these literals that explore different execution paths in the seed program, and use these new values in two different ways.
In the first method, we substitute the original literals with the new values directly, aiming to test the robustness of the optimizations of the compiler given the new execution path. In the second method, we keep the literals as variables, and accept them as input to a modified version of the seed program, so that the new values can be used to generate workload profiles for testing profile-guided optimizations. Using these profiles, we aim to trigger different optimizations on this modified version of the seed program, and to check their correctness when passed the original literal values as input through differential testing.
Using the first method, we find three different compiler bugs using LLVM’s test suite as seed programs, two of which are the result of our mutations, but neither are optimization bugs. Nevertheless, the first method increases line coverage of GCC by 0.63% at optimization level -O3, and the second by 0.19%. We also test the second method on seed programs generated from CSmith and YARPGen. We find one bug in Clang using a seed program from YARPGen. On a bank of 1000 seed programs from each generator, we observe a line coverage increase in GCC of 0.18% and 0.34% respectively.