Syndetic Assistance Processes (SNAP)

Activity recognition in intelligent environments could play a key role for supporting people in their activities of daily life. Partially observable Markov decision process (POMDP) models have been used successfully, for example, to assist people with dementia when carrying out small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modeling assistance that can deal with uncertainty and utility in a theoretically well-justified manner. Unfortunately, POMDPs usually require a very labour intensive, manual setup procedure. The SNAP project aims to build a knowledge driven method for automatically generating POMDP activity recognition and situated prompting systems for a complex tasks. It starts with a psychologically justified description of the task and the particular environment in which it is to be carried out that can be generated from empirical data. This is then automatically combined with a specification of the available sensors and effectors to build a working prompting system.

SNAP System

The SNAP system is online and can be found here. Sign up and get started! Or, watch the video below to see how it works.


This screencast shows how to create a SNAP instance using our database model.

The following video is an example of a SNAP instance (for tea making) in action. Part of the POMDP belief state (marginals) is shown, along with sensor readings. The video shows an archetypal scenario in which a volunteer was asked to prepare a cup of tea. The participant was instructed to wait for prompts before doing each required behavior, but to respond to all prompts except recognition prompts. Whilst we acknowledge that the design of appropriate prompts (in terms of both modality and content) is crucial to the development of an effective practical system for supporting people with dementia, in this case our prompts were chosen only to be recognizable by our participants (i.e. as recall, recognition or affordance reminders). This is filmed in the Ambient Kitchen at Culture Lab