William
Callaghan,
Master’s
candidate
David
R.
Cheriton
School
of
Computer
Science
In this thesis, we present and evaluate a framework for combining machine learning algorithms, crowd workers, and experts in the classification of heart sound recordings. The development of a hybrid human-machine framework for heart sound recordings is motivated by the past success in utilizing human computation to solve problems in medicine as well as the use of human-machine frameworks in other domains. We describe the methods that decide when and how to escalate the analysis of heart sound recordings to different resources and incorporate their decision into a final classification. We present and discuss the results of the framework which was tested with a number of different machine classifiers and a group of crowd workers from Amazon’s Mechanical Turk. We also provide an evaluation of how crowd workers perform in various different heart sound analysis tasks, and how they compare with machine classifiers. In addition, we investigate how machine and human analysis are effected by different types of heart sounds and provide a strategy for involving experts when these methods are uncertain. We conclude that the use of a hybrid framework is a viable method for heart sound classification.