Investigating the Effect of Defect Co fix on Quality Assurance Resource Allocation A Search based Approach
Authors -
Hadi, Hemmati;
Meiyappan, Nagappan and
Ahmed, E. Hassan
Venue -
IAccepted in Journal of Systems and Software, November 2014
Related Tags -
Abstract -
Allocation of resources to pre-release Quality Assurance (QA) tasks, such as source code analysis, peer review, and testing, is
one of the challenges faced by a software project manager. The goal is to find as many defects as possible with the available
QA resources prior to the release. This can be achieved by assigning more resources to the more defect-prone artifacts, e.g., components, classes, and methods. The state-of-the-art QA resource allocation approaches predict the defect-proneness of an
artifact using the historical data of different software metrics, e.g., the number of previous defects and the changes in the artifact.
Given a QA budget, an allocation technique selects the most defect-prone artifacts, for further investigation by the QA team.
While there has been many research efforts on discovering more predictive software metrics and more effective defect prediction
algorithms, the cost-effectiveness of the QA resource allocation approaches has always been evaluated by counting the number
of defects per selected artifact. The problem with such an evaluation approach is that it ignores the fact that, in practice, fixing a
software issue is not bounded to an artifact under investigation. In other words, one may start reviewing a file that is identified as
defect-prone and detect a defect, but to fix the defect one may modify not only the defective part of the file under review, but also
several other artifacts that are somehow related to the defective code (e.g., a method that calls the defective code). Such co-fixes
(fixing several defects together) during analyzing/reviewing/testing of an artifact under investigation will change the number of
remaining defects in the other artifacts. Therefore, a QA resource allocation approach is more effective if it prioritizes the artifacts
that would lead to the smallest number of remaining defects.
Investigating six medium-to-large releases of open source systems (Mylyn, Eclipse, and NetBeans, two releases each), we
found that co-fixes happen quite often in software projects (30-42% of the fixes modify more than one artifact). Therefore, in this
paper, we first introduce a new cost-effectiveness measure to evaluate QA resource allocation, based on the concept of remaining
defects per file. We then propose several co-fix-aware prioritization approaches to dynamically optimize the new measure, based
on the historical defect co-fixes. The evaluation of these approaches on the six releases shows that a) co-fix-aware QA prioritization
approaches improve the traditional defect prediction-based ones, in terms of density of remaining defects per file and b) co-fixaware
QA prioritization can potentially benefit from search-based software engineering techniques.
Preprint -
PDF
BibTex -
@article{Hemmati2014,
author = {Hadi, Hemmati and Meiyappan, Nagappan and Ahmed, E. Hassan},
keyword = {Defect Prediction},
title = {Investigating the Effect of Defect Co fix on Quality Assurance Resource Allocation A Search based Approach},
type = {journal},
venue = {IAccepted in Journal of Systems and Software, November 2014}
}