- 30% of final mark
- To be done individually

- Option A (Literature survey):
- Pick a problem that interests you
- Search the literature for sequential decision making and reinforcement learning approaches to tackle this problem
- Survey and discuss the relative strengths of each approach
- Option B (Empirical
evaluation):

- Pick a problem that interests you.
- Implement and experiment with several sequential decision making and reinforcement learning techniques to tackle this problem.
- Option C (Algorithm design):

- Identify a problem for which there are no satisfying approaches.
- Develop a new sequential decision making and reinforcement learning technique to tackle this problem.
- Analyze theoretically and/or empirically the performance of your technique.
- Option D (Theoretical analysis):

- Identify a problem or decision making/reinforcement learning technique for which the properties (e.g., complexity, performance) are not well understood.
- Analyze the properties of this problem or technique.

- Deadline: June 24

- At most one page
- Which option did you pick?
- Option A (Literature survey):
- What is the problem?
- Cite 8 to 12 papers that you plan to survey.

- Option B (Empirical evaluation):
- What is the problem?
- What decision making or reinforcement learning techniques do you plan to experiment with?
- Cite 4 to 8 related papers that you plan to survey.
- Option C (Algorithm design):
- What is the problem?
- Why are there no satisfying approaches?
- What is the intuition behind the new technique that you plan to
develop?

- Cite 4 to 8 related papers that you plan to survey.

- Option D (Theoretical analysis):
- What is the problem or technique that you plan to analyze?
- What properties would you like to analyze/prove about this
problem or technique?

- Cite 4 to 8 related papers that you plan to survey.

- Deadline: August 19

- At most 8 pages

- Option A (Literature survey):
- Introduction
- What is the problem?
- Why is it an important problem?
- Survey

- Summarize the range of techniques by highlighting their strengths and weaknesses (i.e., the 8-12 papers that you read)
- Tip: this summary should not be a laundry list of techniques with an independent paragraph for each technique
- Suggestion: organize your summary based on desirable
properties of the techniques

- Analysis:
- What is the state of the art?
- Any open problem?
- Conlusion

- What have you learned?

- What future research do you recommend?
- Option B (Empirical evaluation):
- Introduction
- What is the problem?

- Why is it an important problem?

- Techniques to tackle the problem

- Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Brief description of the techniques chosen and why

- Empirical evaluation

- Compare empirically the techniques for complexity, performance, ease of use, etc.
- Conclusion:
- What is the best technique?

- Is any technique good enough to declare the problem solved?
- What future research do you recommend?
- Option C (Algorithm design):
- Introduction
- What is the problem?
- Why can't any of the existing techniques effectively tackle this problem?
- What is the intuition behind the technique that you have
developed?

- Techniques to tackle the problem

- Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Describe the technique that you developed
- Brief description of the existing techniques that you will
compare to

- Evaluation

- Analyze and compare (empirically or theoretically) your new approach to existing approaches
- Conclusion:
- Can your new technique effectively tackle the problem?
- What future research do you recommend?
- Option D (Theoretical analysis):
- Introduction
- What is the problem or technique?
- What properties did you analyze/prove about this problem or
technique?

- Analysis

- Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Describe the analysis performed

- Conclusion:
- What have you discovered about the technique analyzed?
- What future research do you recommend?