This project aims to build an emotionally intelligent cognitive assistant to engage and help older adults with Alzheimer's disease (AD) to complete activities of daily living (ADL) more independently. Our new system combines two research streams. First, the development of cognitive assistants with artificially intelligent controllers using partially observable Markov decision processes (POMDPs, see the COACH project. Second, a model of the dynamics of emotion and identity call Affect Control Theory that arises from the sociological literature on culturally shared sentiments (see the BayesACT project. We have built a prototype assistive technology that combines the two. Work is in progress to understand further the emotional interaction between person's with Alzheimer's and automated prompting systems, and to refine the system and test it. Emotional or affective reasoning is one of the last major hurdles to overcome before intelligent cognitive assistants become widespread.

  • Luyuan Lin
  • Jesse Hoey
  • Aarti Malhotra
  • Celia Yu
  • Stephen Czarnuch
These videos show an actor (in fact it is Luyuan Lin, the student who built the system) interacting with the emotionally aware handwashing system.
The system starts with the identities for client as shown (Elder with EPA of [1.67,0.01,-1.03] or Lonesome Elder with EPA of [[-0.64, -0.43, -1.81], significantly less "good", powerful and active than "elder"). The system has an identity of "assistant" (EPA [1.51,0.51,0.45]) except for in the last video where it is a "bossy assistant" (EPA [-0.89,0.52,0.53], significantly less good).
The system uses a Kinect camera and Stephen Czarnuch's hand tracker.
More details can be found by reading the following papers: