Please note: This master’s thesis presentation will take place in DC 2314 and online.
Gareema Ranjan, Master’s candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Shane McIntosh
Software build tools automate the transformation of source code into deliverables. Since developers invoke builds multiple times per day, the performance of a build tool directly impacts developer productivity. Motivated by the potential of productivity improvements, artifact-based build tools have emerged to accelerate builds. Despite their advantages, recent work shows that a considerable proportion of projects that adopt these tools later downgrade to others. While prior work has explained the rationale of build tool downgrades, the cost of these downgrades in terms of build duration and computational resource usage is not well understood. Without such an understanding, software teams may make under-informed decisions about adopting or abandoning build tools.
In this thesis, we conduct an empirical study of the performance penalties associated with the downgrade of build tools in the Kubernetes project, focusing on its downgrade from an artifact-based build tool (Bazel) to a language-specific solution (Go Build). We reproduce and analyze full and incremental builds of change sets during the period leading up to the downgrade event.
Our results show that, on the one hand, Bazel builds are significantly shorter than Go Build ones, specifically 23.06–38.66% shorter for full builds and up to 75.19% shorter for incremental builds. On the other hand, Bazel builds impose a larger memory footprint than Go Build (81.42–351.07% more for full builds and 118.71–218.22% more for incremental builds) and greater CPU load at parallelism settings beyond eight for full builds and beyond one for incremental builds. To understand the financial impact of abandoning Bazel, we further analyze the costs of resource consumption during continuous integration builds. We find that Bazel tends to be less costly than Go Build, with the estimated additional financial costs of downgrading ranging from 22.62–39.14% for full builds and reaching up to 75.92% for incremental builds. When examining the generalizability of our findings by replicating our Kubernetes study on smaller projects, we observe that build tool downgrades have a smaller impact on build durations for codebases smaller than Kubernetes; however, artifact-based build tools consistently impose a larger memory footprint than their replacements.
We conclude that abandoning artifact-based build tools, despite perceived maintainability benefits, tends to incur considerable performance costs for large projects. This trade-off between tool complexity and efficiency highlights the need for informed decision-making and lays the groundwork for future research to develop tools that support balancing these trade-offs and more pragmatic decision-making about build tool adoption.
To attend this master’s thesis presentation in person, please go to DC 2314. You can also attend virtually on Zoom.