OverviewThis project aims to build an emotionally intelligent cognitive assistant to engage and help older adults with Alzheimer's disease (AD) to complete activities of daily living (ADL) more independently. Our new system combines two research streams. First, the development of cognitive assistants with artificially intelligent controllers using partially observable Markov decision processes (POMDPs, see the COACH project. Second, a model of the dynamics of emotion and identity call Affect Control Theory that arises from the sociological literature on culturally shared sentiments (see the BayesACT project. We have built a prototype assistive technology that combines the two. Work is in progress to understand further the emotional interaction between person's with Alzheimer's and automated prompting systems, and to refine the system and test it. Emotional or affective reasoning is one of the last major hurdles to overcome before intelligent cognitive assistants become widespread.
- Luyuan Lin
- Jesse Hoey
- Aarti Malhotra
- Celia Yu
- Stephen Czarnuch
VideosThese videos show an actor (in fact it is Luyuan Lin, the student who built the system) interacting with the emotionally aware handwashing system.
The system starts with the identities for client as shown (Elder with EPA of [1.67,0.01,-1.03] or Lonesome Elder with EPA of [[-0.64, -0.43, -1.81], significantly less "good", powerful and active than "elder"). The system has an identity of "assistant" (EPA [1.51,0.51,0.45]) except for in the last video where it is a "bossy assistant" (EPA [-0.89,0.52,0.53], significantly less good).
The system uses a Kinect camera and Stephen Czarnuch's hand tracker.
PapersMore details can be found by reading the following papers:
- Alexandra Konig, Linda E. Francis, Jyoti Joshi, Julie M Robillard and Jesse Hoey Journal of Rehabilitation and Assistive Technologies Engineering, 2017 (bibtex)
- Alexandra Konig, Aarti Malhotra, Jesse Hoey and Linda E. Francis Proc. International Conference on Pervasive Computing Technologies for Healthcare, Cancun, Mexico, 2016 (bibtex)
- Alexandra Konig, Linda E. Francis, Aarti Malhotra and Jesse Hoey Proc. International Conference on Pervasive Computing Technologies for Healthcare, Cancun, Mexico, 2016 (bibtex)
- Aarti Malhotra, Jesse Hoey, Alexandra Konig and Sarel van Vuuren Proc. International Conference on Pervasive Computing Technologies for Healthcare, Cancun, Mexico, 2016 (bibtex)
- Luyuan Lin, Stephen Czarnuch, Aarti Malhotra, Lifei Yu, Tobias Schroeder and Jesse Hoey Proc. of International Workconference on Ambient Assisted Living (IWAAL), Belfast, UK, 2014 (bibtex)
- Aarti Malhotra, Lifei Yu, Tobias Schroeder and Jesse Hoey Proc. AAAI Workshop on artificial intelligence applied to assistive technologies and smart environments, Austin, TX, 2015 (bibtex)