NIPS 2010 Workshop on Machine Learning for Assistive Technologies (MLAT-2010)

Schedule

INVITED SPEAKERS

• Prasad Tadepalli, Oregon State University
  Title: A Decision-Theoretic Framework for Assistive Technologies
  Abstract:
  The potential of assistive technologies to transform the lives of both able and disabled people cannot be overestimated. In this talk, we describe a decision-theoretic framework that captures the general problem of optimally assisting a goal-directed user. Since the goals of the users are typically unobserved, a key problem is to infer them from their actions, and balance the uncertainty of the goal with the usefulness of the help offered. We study several instances of this problem as special cases of more general Partially Observable Markov Decision Processes (POMDPs). We apply this framework to a number of domains including the real-world task of folder prediction in Windows, and show that, in spite of the bad worst-case complexity, the performance of myopic heuristics is quite good. We develop a formal model that explains the effectiveness of the myopic heuristics and derive a simple bound on the worst case number of mistakes made relative to an assistant who knows the goals of the user. We suggest open problems and future directions in this line of research to advance the state of the art in assistive technologies. Joint work with Alan Fern.


• Matthai Philipose, Intel Corporation
  Title: Building Machines for Care
  Abstract:
  Assisting the old and infirm with day-to-day tasks is currently exhausting, expensive and manual. Many proposals over the last decade have sought to build machines that could help. In this talk, I will use insights from three generations of sensor-based eldercare systems built and deployed by Intel to understand such machines. The core technology is that of measuring and reporting human behavior based on statistical processing of sensor data. A close look at the details of caregiving reveals "three R's" that such technology must deliver. Recognition must not only be highly reliable, it should preferably apply to a rich variety of behaviors, and do so while providing reassurance of its correctness. I will discuss our efforts and challenges in meeting these goals, with particular focus on sensors (ranging from novel high-density sensing to the use of 3-D cameras), reasoning (ranging across the full spectrum of supervision in learners) and information delivery (ranging from uninterpreted "witness sequences" to providing justification for inference). I will end with a short list of challenge problems that either stand in the way of immediate widespread adoption of these systems or gate dramatic advances in functionality.

ACCEPTED CONTRIBUTIONS

TALKS

• On the design and validation of an intelligent powered wheelchair: Lessons from the SmartWheeler project
  Authors:
  • Joelle Pineau
  • Amin Atrash
  • Robert Kaplow
  • Julien Villemure
  Abstract:
  New-generation, intelligent, powered wheelchairs promise to increase the mobility and freedom of individuals with serious chronic mobility impairments. And while rapid progress continues to be made in terms of the engineering capabilities of robotic wheelchairs, many projects fall short of the target in terms of ease of use, conviviality, and robustness. This paper describes the SmartWheeler, a multi-functional intelligent wheelchair, which leverages state-of-the-art probabilistic techniques for both autonomous navigation and user interaction modeling, to provide a novel robust solution to the problem of assistive mobility. We also discuss the use of standardized evaluation in the development and testing of such technology.
back to top

---



• Learning the Reward Model of Dialogue POMDPs from Data
  Authors:
  • Abdeslam Boularias
  • Hamid R. Chinaei
  • Brahim Chaib-draa
  Abstract:
  Spoken language communication between human and machines has become a challenge in research and technology. In particular, enabling the health care robots with spoken language interface is of great attention. Recently, there has been interest for modelling the dialogue manager of spoken dialogue systems using Partially Observable Markov Decision Processes (POMDPs). With the goal of modelling the dialogue manager of health care robots as dialogue POMDPs, we would like to learn the reward model of dialogue POMDPs from expert's data. In a previous paper work, we used an unsupervised learning method for learning the states, as well as the transition and observation functions of the dialogue POMDPs based on human human dialogues. Continuing our objective of learning the components of dialogue POMDPs from data, we introduce a novel inverse reinforcement learning algorithm for learning the reward function of the dialogue POMDP model. Based on the introduced method, and from an available corpus of data we construct a dialogue POMDP. Then, the learned dialogue policies, based on the learned POMDP, are evaluated. The empirical evaluation shows that the performance of the learned POMDP is higher than expert performance in non, low, and medium noise levels, but the high noise level. At the end, current limita- tions and future directions are addressed
back to top

---



• Transfer Learning for the Labelling Problem
  Authors:
  • G. Englebienne
  • T.L.M. van Kasteren
  • B.J.A. Kröse
  Abstract:
  The collection of real-world data is a continuous problem for researchers investigating healthcare systems. In particular, although it is often relatively straight-forward to collect sets of measurements, the corresponding annotation is typically difficult and expensive to obtain. In this paper, we explore how transfer learning can be used to create person-specific probabilistic models for activity recognition when no annotations are available for that person. We evaluate our approach on three large real-world datasets and show that we achieve good classification performance even when little or no labelled data is available.
back to top

---



• Adaptive Prompting for Intelligent Wheelchairs
  Authors:
  • Pooja Viswanathan
  • James Little
  • Alan Mackworth
  Abstract:
  Cognitive impairments prevent older adults from using powered wheelchairs due to safety concerns, thus reducing their mobility and independence. An intelligent powered wheelchair is proposed to help restore mobility, while ensuring safe navigation. Machine vision and learning techniques are used to help prevent collisions with obstacles, as well as provide navigation assistance through adaptive prompts.
back to top

---



• Considerations for Developing Intelligent Assistive Technologies for Older Adults with Dementia: Lessons Learned Through Real-world Applications
  Authors:
  • Jennifer N. Boger
  • Tim Adlam
  • Jasper Snoek
  • Alex Mihailidis
  Abstract:
  Older adults with dementia represent a rapidly growing demographic that requires much supervision and care. Intelligent assistive technologies hold great promise as a way to support both a person with dementia and his or her caregivers. If these technologies are to be effective, it is imperative that developers have a good understanding of the needs and abilities of this population to ensure interventions are appropriate. This paper presents lessons learned through real-world applications of prototype devices and outlines considerations for developers who are looking to create supportive technologies for people with dementia.
back to top

---



• The Research Challenges of ClassmateAssist: A Personal and Physical Math Coin Tutoring System
  Authors:
  • Georgios Theocharous
  • Nicholas Butko
  • Matthai Philipose
  Abstract:
  Many elementary mathematics teachers believe that learning improves significantly when students are instructed with physical objects such as coins, called manipulatives. Unfortunately, teaching with manipulatives is a time consuming process that is best with personalized 1-to-1 tutoring. In this paper, we explore the research challenges and solutions of an automated physical and personal tutoring solution.
back to top

---



• A Utility and Context Based Framework for Addressing the Machine Learning Challenges of Detecting Rare but Important Clinical Events
  Authors:
  • Holly Jimison
  • Misha Pavel
  Abstract:
  Designing systems for home use by older adults and those with chronic conditions that impair cognitive or physical function can be especially challenging. Machine learning techniques are critical for categorizing patient state and understanding how best to intervene in an autonomous setting. One of the primary challenges that arises with many clinical applications is that the important events to detect can be quite rare, for example, falls in the home. This means that we often have minimal or no examples at all of the events to use in the training of standard classification approaches. This paper describes the challenges of applying machine learning techniques to home monitoring clinical data, as well as a framework for integrating environmental context and utilities associated with event classes to address the issue of detecting important but rare events.
back to top

---



• Robust activity recognition for assistive technologies: Benchmarking ML techniques
  Authors:
  • Ricardo Chavarriaga
  • Jose del R. Millan
  • Hesam Sagha
  • Hamidreza Bayati
  • Paul Lukowicz
  • David Bannach
  • Daniel Roggen
  • Kilian Förster
  • Alberto Calatroni
  • Gerhard Tröster
  • Gerold Hölzl
  • Alois Ferscha
  • Marc Kurz
  Abstract:
  An increasing need for healthcare provision and assistive technologies (AT) calls for the development of machine learning techniques able to cope with the variability inherent to real-world deployments. In the particular case of activity recognition applications sensor networks may be prone to changes at different levels rang- ing from sensor data variability to network reconfiguration. Robust methods are required to deal with those changes providing graceful degradation upon failure or self-configuration and adaptation capabilities that ensure their proper operation for long periods of time. Currently there is a lack of common tools and datasets that allow for replicable and fair comparison of different recognition approaches. We introduce a large database of human daily activities recorded in a sensor-rich environment. The database provides large amount of instances of the recorded activities using a significant number of sensors. In addition, we reviewed some of the techniques that have been proposed to cope with changes in the system, including missing data, sensor location/orientation change, as well as the possibility to exploit data from unknown discovered sensors. These techniques have been tested in the aforementioned datasets showing its suitability to emulate different sensor network configurations and recognition goals.
back to top

---



• Automating the Calibration of a Neonatal Condition Monitoring System
  Authors:
  • Chris Williams
  • Ioan Stanculescu
  Abstract:
  Condition monitoring of premature babies in intensive care can be carried out using a Factorial Switching Linear Dynamical System (FSLDS) [19]. A crucial part of training the FSLDS is the manual calibration stage, where an interval of normality must be identified for each baby that is monitored. In this paper we replace this manual step by using a classifier to predict whether an interval is normal or not. We show that the monitoring results obtained using automated calibration are almost as good as those using manual calibration.

POSTERS

• Learning mixed acoustic/articulatory models for disabled speech
• Authors:
  • Frank Rudzicz
• This paper argues that automatic speech recognition (ASR) should accommodate dysarthric speech by incorporating knowledge of the production characteristics of these speakers. We describe the acquisition of a new database of dysarthric speech that includes aligned acoustics and articulatory data obtained by electromagnetic articulography. This database is used to train theoretical and empirical models of the vocal tract within ASR which are compared against discriminative models such as neural networks, support vector machines, and conditional random fields.Results show significant improvements in accuracy over the baseline through the use of production knowledge.
back to top

---



• An Uncued Brain-Computer Interface Using Reservoir Computing
  Authors:
  • Pieter-Jan Kindermans
  • Pieter Buteneers
  • David Verstraeten
  • Benjamin Schrauwen
  Abstract:
  Brain-Computer Interfaces are an important and promising avenue for possible next-generation assistive devices. In this article, we show how Reservoir Computing -- a computationally efficient way of training recurrent neural networks -- combined with a novel feature selection algorithm based on Common Spatial Patterns can be used to drastically improve performance in an uncued motor imagery based Brain-Computer Interface (BCI). The objective of this BCI is to label each sample of EEG data as either motor imagery class 1 (e.g. left hand), motor imagery class 2 (e.g. right hand) or a rest state (i.e., no motor imagery). When comparing the results of the proposed method with the results from the BCI Competition IV (where this dataset was introduced), it turns out that the proposed method outperforms the winner of the competition.
back to top

---



• A new model based on task recognition and monitoring for the development of sensory substitution assistive systems for the visually impaired
  Authors:
  • Pantelis Elinas
  • Yi Li
  • Lochana Perera
  Abstract:
  According to the World Health Organization more than 314 million people world- wide suffer from some form of visual impairment with 87% of them living in the developing world. Clearly, there exists a need for the development of cost- effective assistive technologies for improving the quality of life for the visually impaired. Sensory substitution systems aim to replace one sensory signal, e.g., vision, with another, e.g., haptic, delivered via vibrotactile or electrotactile stimulation. Although much progress has been made towards the development of such systems some of which have been made available commercially their capabilities and usability are still far from desired. In this paper, we survey recent advances in sensory substitution with a focus on developing assistive devices for the visually impaired, identify a number of roadblocks to the development of more advanced systems and propose a new development model utilizing state-of-the-art machine learning techniques. We believe that the proposed model will allow us to lift some of the obstacles preventing the mass adaptation of such assistive devices.
back to top

---



• Task Assistance for Persons with cognitive Disabilities (TAPeD)
  Authors:
  • Christian Peters
  • Thomas Hermann
  • Sven Wachsmuth
  Abstract:
  TAPeD is a project at the Cognitive Interaction Technology, Center of Excellence (CITEC) at Bielefeld University with the aim to develop an automatic prompting system in the healthcare domain. In comparison to systems applied in individual user's homes to prolong the user's independence in everyday life, we aim to develop a system for a residential home where persons with different cognitive disabilities live together and share the same system. We cooperate with Haus Bersaba, a residential home belonging to v. Bodelschwinghsche Stiftungen Bethel which is a care facility in Bielefeld, Germany. Our user group has problems fulfilling Activities of Daily Living (ADLs), in particular in brushing teeth. We describe the progress of development towards an automatic prompting system assisting in brushing teeth and give an overview of our project from a machine learning perspective.
back to top

---



• Activity Recognition for Users of Rolling Walker Mobility Aids
  Authors:
  • Mathieu Sinn
  • Pascal Poupart
  Abstract:
  We present Smart Walkers, a comprehensive approach to enhancing independent and safe mobility of elderly people. The idea of the Smart Walkers project is to equip rolling walker mobility aids with sensors and actuators. The goal is to assist users, caregivers and clinicians, e.g., by monitoring the physical and mental conditions of the user, detecting risks of falling, assessing the effectiveness of therapeutic interventions, and providing active navigation assistance. The key problem in building the Smart Walkers technology is the ability to recognize the user activity from the stream of sensor measurements. In this paper we present supervised and unsupervised machine learning algorithms for this purpose and discuss their performance on real user data. We find that the best results are obtained for Conditional Random Fields with feature functions based on thresholding, achieving an accuracy of 85-90%.
back to top

---



• Probabilistic Cursor Trajectory Prediction via Inverse Optimal Control
  Authors:
  • Brian D. Ziebart
  • Anind Dey
  • J. Andrew Bagnell
  Abstract:
  Many tasks in people's everyday lives can be viewed as control problems where assistive technologies could intervene in various ways to improve task performance. Understanding people's goals and predicting the actions they intend to employ to achieve those goals is important for choosing appropriate interventions. In this paper, we present a novel inverse optimal control approach for learning and predicting those intentions and future actions. We are particularly motivated by assistive technologies for cursor-based input in this work. Under our approach, cursor trajectories are assumed to be stochastically generated from a continuous control process and the parameters of that process that best explain a s goal-directed cursor motions are learned. The resulting predictions are then efficient for real-time intervention selection and personalized to the individual.
back to top

---



• Toward a system for stroke rehabilitation user centred
  Authors:
  • Roger Luis Velazquez
  • Enrique Sucar
  Abstract:
  This paper describes a design of a system rehabilitation games-based therapy, whose goal is monitor the user, provide recommendations on the parameters of activities and adaptation to user performance. We believe that a rehabilitation system coupled to the human behavior is necessary to improve the potential for rehabilitation. To accomplish this, we should consider at least two essential aspects in the design of a recommendation for a robotic rehabilitation system - physical and emotional. We propose a model that takes into account physical aspects and emotions of the user, and can be adapted to the abilities of the patient with the aim of measuring their learning in his rehabilitation tasks.
back to top

---



• Enhancing Social Interactions of Individuals with Visual Impairments: A Case Study for Assistive Machine Learning
  Authors:
  • Vineeth Balasubramanian
  • Shayok Chakraborty
  • Sreekar Krishna
  • Sethuraman Panchanathan
  Abstract:
  Individuals with visual impairments face serious challenges in experiencing the fundamental privileges of social interactions. The realization of a Social Interaction Assistant (SIA) device for such individuals involves solving several challenging problems in pattern analysis and machine intelligence such as person recognition/tracking, head/body pose estimation, posture/gesture recognition, expression recognition, and human-object interaction recognition on a combination of wearable and ubiquitous computing platforms. This work presents sample machine learning contributions that have been made as part of the development of such a SIA device, including integrated face localization and detection, user-conformal confidence measures and online active learning.
back to top

---



• Self Organizing Maps for Affective State Detection
  Authors:
  • Bert Arnrich
  • Cornelia Kappeler-Setz
  • Roberto La Marca
  • Gerhard Troester
  • Ulrike Ehlert
  Abstract:
  In this contribution we present two experimental scenarios in which we employed Self Organizing Maps (SOMs) to detect affective states. The first scenario is related towards designing a ``Personal Stress Prevention Assistant'': we summarize our efforts to detect affective information related to stress in the posture channel. We show that a person-independent discrimination of stress from cognitive load is feasible when using data from a pressure mat mounted on a seat. The second scenario is embedded towards assisting patients with manic depression: we present preliminary results in detecting emotions from voice data. Our findings illustrate that a person-dependent discrimination of emotions from voice data seems feasible and that a general model might be appropriate to discriminate high and low levels of arousal.
back to top

---



• Robust Local Video Event Detection for Action Recognition
  Authors:
  • Amir-Hossein Shabani
  • John Zelek
  • David A. Clausi
  Abstract:
  Human action recognition is an important component of elderly activity analysis for assisted living. Actions can be represented using a set of local video events. Using Poisson filtering, a novel robust video event detection approach is devel- oped. This approach is consistent with the human's biological vision and motion perception models. The extracted video events show high precision rate and high reproducibility score under different view perspectives and scale changes. In a standard bag-of-words framework, these events are shown to improve the average accuracy in the recognition of ten different actions in the Weizmann data set.
back to top

---



• SN̈AP: Syndetic Assistance Processes
  Authors:
  • Jesse Hoey
  • Thomas Ploetz
  • Dan Jackson
  • Andrew Monk
  • Cuong Pham
  • Patrick Olivier
  Abstract:
  Activity recognition in intelligent environments could play a key role for support- ing people in their activities of daily life. Partially observable Markov decision process (POMDP) models have been used successfully, for example, to assist people with dementia when carrying out small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modeling assistance that can deal with uncertainty and utility in a theoretically well-justified manner. Unfortunately, POMDPs usually require a very labor intensive, manual setup procedure. This paper describes a knowledge driven method for automatically generating POMDP activity recognition and context sensitive prompting systems for a complex tasks.We call the resulting POMDP a SN̈AP (SyNdetic Assistance Process). The method starts with a psychologically justified (syndetic) description of the task and the particular environment in which it is to be carried out that can be generated from empirical data. This is then combined with a specification of the available sensors and effectors to build a working prompting system that tracks a person's activities and learns their abilities by using sensor data as evidence in the context of the SN̈AP POMDP. The method is illustrated by building a system that prompts through the task of making a cup of tea in a real-world kitchen.
back to top

---