CS886 Schedule and Readings

• Russell and Norvig, Artificial Intelligence: A Modern Approach
• Puterman, Markov Decision Processes
  
• Sutton and Barto, Reinforcement learning
• Mitchell, Machine Learning
• Bertsekas and Tsitsiklis, Neuro-dynamic programming
  

• Jan 4
  
• Course overview
• Lecture slides
  
• Jan 6
• Introduction to probabilistic reasoning
• Bayesian networks
• Variable Elimination
• Lecture slides
  
• Nevin Lianwen Zhang and David Poole, ``A simple approach to Bayesian network computations'', Proceedings of the Tenth Biennial Canadian Artificial Intelligence Conference (AI-94), Banff, May 1994, 171-178.
• Jan 11
  
• Message passing and loopy belief propagation
• Markov networks
  
• Kevin Murphy, Yair Weiss and Michael Jordan, "Loopy Belief Propagation for Approximate Inference: an Empirical Study"
• Yair Weiss, Correctness of Local Probability Propagation in Graphical Models with Loops(2000)
• Jan 13
• Sampling techniques
  
• [Presenter: Dimitri Mostinski] Michael Shwe and Gregory Cooper, An empirical analysis of likelihood-weighting simulation on a large multiply connected belief network, Computers and Biomedical Research 24, p.  453-475 (1991) [Download this paper from the e-journal's section of the UW library website]
  
• Jan 18
• Learning Bayesian networks
  
• [Presenter: Sameeh Ullah] David Heckerman, A Tutorial on Learning with Bayesian Networks
• Jan 20
• Probabilistic constraint satisfaction
• Greg A. Keim and Noam Shazeer and Michael L. Littman and Sushant Agarwal and Catherine M. Cheves and Joseph Fitzgerald and Jason Grosland and Fan Jiang and Shannon Pollard and Karl Weinmeister. Proverb: The Probabilistic Cruciverbalist.1999. in Proceedings of the Sixteenth National Conference on Artificial Intelligence, pages 710--717.
• [Presenter: Leila Chinaei] Noam M. Shazeer and Michael L. Littman and Greg A. Keim. Solving Crossword Puzzles as Probabilistic Constraint Satisfaction.1999. in Proceedings of the Sixteenth National Conference on Artificial Intelligence, pages 156--162.
• Jan 25
• Influence diagrams or decision networks
• Lecture Slides
  
• [Presenter: Omar Khan] John Breese and David Heckerman, Topics in Decision-Theoretic Troubleshooting: Repair and Experiment
• [Presenter: Radoslav Radoulov] Eric Horvitz, Jack Breese, David Heckerman, David Hovel, Koos Rommelse, The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users
• Jan 27
• Dynamic Bayesian networks and hidden Markov models
• [Presenter: Laurent Charlin] Padhraic Smyth, David Heckerman, Michael Jordan, Probabilistic Independence Networks for Hidden Markov Probability Models (1996)
• Feb 1
  
• Robot localisation
• [Presenter: Martin Talbot] D. Fox, W. Burgard, F. Dellaert, and S. Thrun. Monte carlo localization: Efficient position estimation for mobile robots.In Proceedings of the National Conference on Artificial Intelligence (AAAI), Orlando, FL, 1999. AAAI. (There is also a low resolution PDF version)
• [Presenter: Nabeel Ahmed] S. Thrun, M. Bennewitz, W. Burgard, F. Dellaert, D. Fox, D. Haehnel, C. Rosenberg, N. Roy, J. Schulte, and D. Schulz. MINERVA: A second generation mobile tour-guide robot
• Feb 3
  
• Intro to Markov decision processes
• Feb 8
  
• MDP algorithms and algebraic decision diagrams
  
• [Presenter: Luke Tanur] Jesse Hoey , Robert St-Aubin, Alan Hu and Craig Boutilier, SPUDD: Stochastic Planning using Decision Diagrams
• [Presenter: ___________] Schuurmans, D. and Patrascu, R. (2001) Direct value-approximation for factored MDPs. In Advances in Neural Information Processing 14 (NIPS-2001).
• Feb 10
  
• Reinforcement learning
• Richard S. Sutton and Andrew G. Barto, Reinforcement learning: an introduction (Chapters 5 and 6)
• [Presenter: Narges Simjour]  Watkins and Dayan, Q-learning, Machine learning (8) p. 279-292 [Download this paper from the e-journal's section of the UW library website]
• Feb 15
• Neuro-dynamic programming
• [Presenter: Tyrel Russel] Gerald Tesauro, Temporal difference learning and TD-Gammon
• Feb 17
  
• Partially Observable Markov Decision Processes
• Tony Cassandra, POMDPs for dummies
• [Presenter: Allan Caine] Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra.  Planning and acting in partially observable stochastic domains. Artificial Intelligence, Volume 101, pp. 99-134, 1998
• Feb 22
• Point-based value iteration
• J. Pineau, G. Gordon & S. Thrun "Point-based value iteration: An anytime algorithm for POMDPs''. International Joint Conference on Artificial Intelligence (IJCAI). Acapulco, Mexico. pp. 1025-1032. Aug. 2003
• [Presenter: David Tausky] Matthijs T. J. Spaan and Nikos Vlassis. A point-based POMDP algorithm for robot planning. In Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2399–2404, New Orleans, Louisiana, 2004.
• Feb 24
• No lecture: UW reading day
• Mar 1
• Spoken dialog systems
• [Presenter: Kevin Regan] N. Roy, J. Pineau & S. Thrun. "Spoken Dialog Management Using Probabilistic Reasoning''. Association for Computational Linguistics (ACL). Hong Kong, Oct. 2000
• Mar 3
• Helicopter control
• PEGASUS: A policy search method for large MDPs and POMDPs, Andrew Y. Ng and Michael Jordan. In Uncertainty in Artificial Intelligence, Proceedings of the Sixteenth Conference, 2000.
• [Presenter: David Yeung] Autonomous Helicopter Flight via Reinforcement Learning  Andrew Y. Ng, H. Jin Kim, Michael I. Jordan, Shankar Sastry
• Inverted autonomous helicopter flight via reinforcement learning, Andrew Y. Ng, Adam Coates, Mark Diel, Varun Ganapathi, Jamie Schulte, Ben Tse, Eric Berger and Eric Liang. In International Symposium on Experimental Robotics, 2004.
• Mar 8
• Bayesian reinforcement learning
• [Presenter: Andy Chiu] Richard Dearden, Bayesian Q-learning,
  
• Michael Duff, Design for an Optimal Probe, Proceedings of the 20th International Conference on Machine Learning, 2003.
• Mar 10
  
• Stochastic SAT
• Michael L. Littman, Stephen M. Majercik, and Toniann Pitassi. Stochastic Boolean satisfiability. To appear in Journal of Automated Reasoning, special issue on Satisfiability in the Year 2000.
• [Presenter: Eduardo Barrenechea] Stephen M. Majercik and Michael L. Littman. Contingent planning under uncertainty via stochastic satisfiability. In Proceedings of the Sixteenth National Conference on Artificial Intelligence, pages 549-556, 1999
• Mar 15
• Multi-agent Systems and Distributed AI
• Nikos Vlassis, A Concise Introduction to Multiagent Systems and Distributed AI (Chapters 1, 2, 3, 4 and 8)
  
• Mar 17
• Coordination in multi-agent systems
• [Presenter: Mohammad Munawar] Georgios Chalkiadakis and Craig Boutilier: "Coordination in Multiagent Reinforcement Learning: A Bayesian Approach"
• Mar 22
• Partially Observable Stochastic Games
• [Presenter: Maryam  Shokri] Eric Hansen, Daniel Bernstein and Shlomo Zilbernstein, Dynamic Programming for Partially Observable Stochastic Games
• Mar 24
• Course Summary
• Mar 29
• Project presentations
• Allan Caine
• Mohammad Ahmad Munawar
• Kevin Regan
• Andy Chiu
  
• Mar 31
  
• Project presentations
• Laurent Charlin
• Maryam Shokri
• David Yeung
• Tyrel Russell
• Dimitri Mostinski
  
• Apr 5
• Project presentations
• Martin Talbot
• Omar Zia Khan
• Radoslav Radoulov
• Sameeh Ullah
• Narges Simjour
  
• Apr 7
• Project presentations
• Leila Chinaei
• Nabeel Ahmed
• David Tausky
• Luke Tanur
• Eduardo Barreneche