Sample problems included with SymbolicPerseus --------------------------------------------- Author: Pascal Poupart (ppoupart@cs.uwaterloo.ca) * POMDP problems * Syntax to encode POMDP problems POMDP problems -------------- The following problems are provided with SymbolicPerseus: * Coffee: classic coffee problem that illustrates how to encode a simple factored POMDP. * Handwashing: 4 versions of the handwashing problem 1) problems/handwashing/cppo3.txt - 180 states, 6 observations, 6 actions - reduced version of the handwashing problem 2) problems/handwashing/ver26_15_unstructured.txt - 50,181,120 states, 12 observations, 20 actions - Reference: A Decision-Theoretic Approach to Task Assistance for Persons with Dementia Jennifer Boger, Pascal Poupart, Jesse Hoey, Craig Boutilier, Geoff Fernie, and Alex Mihailidis In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 1293-1299, Edinburgh, Scotland, 2005 3) problems/handwashing/cppo37c_unstructured.txt - 207,360 states, 198 observation, 26 actions - Reference: Assisting Persons with Dementia during Handwashing Using a Partially Observable Markov Decision Process Jesse Hoey, Axel von Bertoldi, Pascal Poupart, and Alex Mihailidis In Proceedings of the International Conference on Vision Systems (ICVS), Biefeld, Germany, 2007. 4) problems/handwashing/cppo37i_unstructured.txt - 207,360 states, 198 observation, 26 actions - Reference: Automated Handwashing Assistance for Persons with Dementia Using Video and a Partially Observable Markov Decision Process Jesse Hoey, Pascal Poupart, Axel von Bertoldi, Tamy Craig, Craig Boutilier and Alex Mihailidis Computer Vision and Image Understanding (CVIU), accepted January 2009. * Networks: system administration network problems (cycle and 3legs). The network problems can be generated by running the matlab commands >> pomdp3legsFlatAct(N) >> pomdpCycleFlatAct(N) where N is the number of machines in the network. A network of N machines gives rise to a POMDP of 2^N states. Alternatively, SPUDD encodings of the 3leg and cycle problems are provided in the problems/network directory for networks of 4, 7, 10, 13, 16, 19, 22 and 25 machines. References for the network problems: VDCBPI: an Approximate Scalable Algorithm for Large Scale POMDPs Pascal Poupart and Craig Boutilier In Advances in Neural Information Processing Systems 17 (NIPS), pages 1081-1088, Vancouver, BC, 2004 Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes Pascal Poupart Ph.D. thesis, Department of Computer Science, University of Toronto, Toronto, 2005 Syntax ------ The file problems/SYNTAX describes the formal grammar used to encode the above factored POMDP problems. The format is a subset of the SPUDD format originally developed by Jesse Hoey (http://www.computing.dundee.ac.uk/staff/jessehoey/spudd/). For a simple example that illustrates this syntax, see problems/coffee/coffee3po.txt, which encodes the classic coffee problem. POMDP problems encoded with this syntax can be read by the symbolicPerseus package using the matlab command: >> parsePOMDP('path/problemName') The class javaClasses/ParseSPUDD encodes the parser used by symbolicPerseus. It is possible to convert small POMDP problems to the Cassandra format by typing the following matlab commands: >> ddPOMDP = parsePOMDP('path/problemName'); >> convertPOMDP2CassandraFormat(ddPOMDP,'path/fileName'); Note that the Cassandra format requires the enumeration of all states (flat format) and therefore may not be suitable for large problems. Depending on the amount of RAM available it may not be possible to convert problems with more than 10,000 states.