Nalin Chhibber, Master’s candidate
David R. Cheriton School of Computer Science
The traditional process of building interactive machine learning systems can be viewed as a teacher-learner interaction scenario where the machine-learners are trained by one or more human-teachers. In this work, we explore if teachable AI agents can reliably learn from human-teachers through conversational interactions, how this teaching process affects teacher’s performance in the task, and their trust on the agent. We introduce a teachable agent, named Kai, that learns to classify news articles while also guiding the teaching process through conversational interventions.
In a three-part study, where several crowdworkers individually teach Kai, we investigate whether this Learning by Teaching approach creates reliable machine learners, improves Turkers’ performance and leads to trustable AI agents that crowdworkers would use. We present and discuss the results of the underlying classifier built after conversational interactions with other text classification algorithms. We also provide an evaluation of how crowdworkers perform a text classification before and after interacting with a teachable agent. Finally, we investigate the notion of trust that crowdworkers would put on teachable agents in terms of delegating the work involving monetary compensation. Together, our results demonstrate the benefits of Learning by Teaching approach, in terms of the performance of the AI agent, the crowdworkers, and the dynamics of trust built from the teacher-learner interaction.
200 University Avenue West
Waterloo, ON N2L 3G1