PhD Defence • Software Engineering • Tacit Inefficiencies and Barriers in Continuous Integration

Please note: This PhD defence will take place DC 2314.

Nimmi Rashinika Weeraddana, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Shane McIntosh

Continuous Integration (CI) is the heartbeat of a software project. CI enables team members to validate each change set through an automated cycle (i.e., a CI build) that compiles and tests the project’s source code. Although adoption of CI improves team productivity and software quality, these benefits come at a cost. As projects evolve, the complexity of CI pipelines tends to increase, introducing potential inefficiencies (i.e., prolonged build durations and frequent build restarts) and barriers (e.g., the specialized expertise required to maintain CI artifacts). Such inefficiencies and barriers waste resources that enable CI.

While inefficiencies and barriers in CI are often explicit, where project teams are cognizant of them, there also exist tacitly accrued inefficiencies and barriers that are not immediately apparent to project teams. In this thesis, we use historical data from a large collection of software projects to perform three empirical studies, focusing on tacit inefficiencies and barriers in CI.

We first present an empirical study that focuses on tacit inefficiencies in the environment (e.g., CircleCI) where CI builds are executed. We observe that (1) CI builds can unexpectedly time out due to issues in the environment, such as network problems and resource constraints, and (2) the history of previous CI build outcomes and anticipation of clusters of consecutive timeouts can provide useful indications to project teams to proactively allocate resources and take preventive measures. Next, we present an empirical study that investigates tacit inefficiencies in CI that stem from dependencies in projects (e.g., npm dependencies). More specifically, CI builds triggered from change sets that update versions of unused dependencies are entirely wasteful because such change sets do not impact the project source code. We find that (1) a substantial amount of CI build time is spent on these wasteful builds, (2) bots that automatically manage dependency updates in projects (e.g., Dependabot) need to consider whether a dependency is used before triggering a build, and (3) to detect and omit such wasteful builds, project teams may adopt our automated approach, Dep-sCImitar, to cut down on this waste.

We then present an empirical study that investigates tacit barriers that are related to the composition of the teams responsible for creating and maintaining CI pipelines, i.e., the DevOps contributors. In particular, we examine the diversity and inclusion of these contributors—a factor that plays a crucial role in CI by influencing collaboration and the overall efficiency of CI pipelines. Our findings show that (1) the perceived ethnic diversity of DevOps contributors is significantly low compared to other contributors, with a similar pattern observed for perceived gender diversity, and (2) the lack of diversity is amplified when considering the intersection of minority ethnicities and genders, calling for enhanced awareness of the lack of diversity among DevOps contributors.