CS886 Spring13 - Course Project

• 30% of final mark
• To be done individually
  

Options

• Option A (Literature survey):
• Pick a problem that interests you
• Search the literature for sequential decision making and reinforcement learning approaches to tackle this problem
• Survey and discuss the relative strengths of each approach
• Option B (Empirical evaluation):
  
• Pick a problem that interests you.
• Implement and experiment with several sequential decision making and reinforcement learning techniques to tackle this problem.
• Option C (Algorithm design):
  
• Identify a problem for which there are no satisfying approaches.
• Develop a new sequential decision making and reinforcement learning technique to tackle this problem.
• Analyze theoretically and/or empirically the performance of your technique.
• Option D (Theoretical analysis):
  
• Identify a problem or decision making/reinforcement learning technique for which the properties (e.g., complexity, performance) are not well understood.
• Analyze the properties of this problem or technique.

Proposal (no mark)

• Deadline: June 24
  
• At most one page
• Which option did you pick?
• Option A (Literature survey):
• What is the problem?
• Cite 8 to 12 papers that you plan to survey.
  
• Option B (Empirical evaluation):
• What is the problem?
• What decision making or reinforcement learning techniques do you plan to experiment with?
• Cite 4 to 8 related papers that you plan to survey.
• Option C (Algorithm design):
• What is the problem?
• Why are there no satisfying approaches?
• What is the intuition behind the new technique that you plan to develop?
  
• Cite 4 to 8 related papers that you plan to survey.
  
• Option D (Theoretical analysis):
• What is the problem or technique that you plan to analyze?
• What properties would you like to analyze/prove about this problem or technique?
  
• Cite 4 to 8 related papers that you plan to survey.

Report

• Deadline: August 19
  
• At most 8 pages

Suggested Structure for the Report

• Option A (Literature survey):
• Introduction
• What is the problem?
• Why is it an important problem?
• Survey
  
• Summarize the range of techniques by highlighting their strengths and weaknesses (i.e., the 8-12 papers that you read)
• Tip: this summary should not be a laundry list of techniques with an independent paragraph for each technique
• Suggestion: organize your summary based on desirable properties of the techniques
  
• Analysis:
• What is the state of the art?
• Any open problem?
• Conlusion
  
• What have you learned?
  
• What future research do you recommend?
• Option B (Empirical evaluation):
• Introduction
• What is the problem?
  
• Why is it an important problem?
  
• Techniques to tackle the problem
  
• Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
• Brief description of the techniques chosen and why
  
• Empirical evaluation
  
• Compare empirically the techniques for complexity, performance, ease of use, etc.
• Conclusion:
• What is the best technique?
  
• Is any technique good enough to declare the problem solved?
• What future research do you recommend?
• Option C (Algorithm design):
• Introduction
• What is the problem?
• Why can't any of the existing techniques effectively tackle this problem?
• What is the intuition behind the technique that you have developed?
  
• Techniques to tackle the problem
  
• Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
• Describe the technique that you developed
• Brief description of the existing techniques that you will compare to
  
• Evaluation
  
• Analyze and compare (empirically or theoretically) your new approach to existing approaches
• Conclusion:
• Can your new technique effectively tackle the problem?
• What future research do you recommend?
• Option D (Theoretical analysis):
• Introduction
• What is the problem or technique?
• What properties did you analyze/prove about this problem or technique?
  
• Analysis
  
• Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
• Describe the analysis performed
  
• Conclusion:
• What have you discovered about the technique analyzed?
• What future research do you recommend?