**40% of final grade**- To be done individually

- Option A (Literature survey):
- Pick a problem that interests you
- Search the literature for reinforcement learning approaches to tackle this problem
- Survey and discuss the relative strengths of each approach

- Option B (Empirical evaluation):
- Pick a problem that interests you.
- Implement and experiment with several reinforcement learning techniques to tackle this problem.

- Option C (Algorithm design):
- Identify a problem for which there are no satisfying approaches.
- Develop a new reinforcement learning technique to tackle this problem.
- Analyze theoretically and/or empirically the performance of your technique.

- Option D (Theoretical analysis):
- Identify a problem or reinforcement learning technique for which the properties (e.g., complexity, performance) are not well understood.
- Analyze the properties of this problem or technique.

**Submit electronically via LEARN by Nov 3 (11:59 pm)**- At most one page
- Which option did you pick?
- Option A (Literature survey):
- What is the problem?
- Cite 8 to 12 papers that you plan to survey.
- Option B (Empirical evaluation):
- What is the problem?
- What reinforcement learning techniques do you plan to experiment with?
- Cite 4 to 8 related papers that you plan to review.

- Option C (Algorithm design):
- What is the problem?
- Why are there no satisfying approaches?
- What is the intuition behind the new reinforcement learning technique that you plan to develop?
- Cite 4 to 8 related papers that you plan to review.

- Option D (Theoretical analysis):
- What is the reinforcement learning problem or technique that you plan to analyze?
- What properties would you like to analyze/prove about this problem or technique?
- Cite 4 to 8 related papers that you plan to review.

- At most 8 pages (excluding references)
- Use the JMLR format: https://www.jmlr.org/format/format.html
- Explain the big picture and any necessary detail
**Submit electronically via LEARN by Dec 10 (11:59 pm)**

- Option A (Literature survey):
- Introduction
- What is the problem?
- Why is it an important problem?

- Survey
- Summarize the range of techniques by highlighting their strengths and weaknesses (i.e., the 8-12 papers that you read)
- Tip: this summary should
**not**be a laundry list of techniques with an independent paragraph for each technique - Suggestion: organize your summary based on desirable properties of the techniques

- Analysis:
- What is the state of the art?
- Any open problem?

- Conclusion
- What have you learned?
- What future research do you recommend?

- Introduction
- Option B (Empirical evaluation):
- Introduction
- What is the problem?
- Why is it an important problem?

- Techniques to tackle the problem
- Brief review of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Brief description of the techniques chosen and why

- Empirical evaluation
- Compare empirically the techniques for complexity, performance, ease of use, etc.

- Conclusion:
- What is the best technique?
- Is any technique good enough to declare the problem solved?
- What future research do you recommend?

- Introduction
- Option C (Algorithm design):
- Introduction
- What is the problem?
- Why can't any of the existing techniques effectively tackle this problem?
- What is the intuition behind the technique that you developed?

- Techniques to tackle the problem
- Brief review of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Describe the technique that you developed
- Brief description of the existing techniques that you will compare to

- Evaluation
- Analyze and compare (empirically or theoretically) your new approach to existing approaches

- Conclusion:
- Can your new technique effectively tackle the problem?
- What future research do you recommend?

- Introduction
- Option D (Theoretical analysis):
- Introduction
- What is the problem or technique?
- What properties did you analyze/prove about this problem or technique?

- Analysis
- Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
- Describe the analysis performed

- Conclusion:
- What have you discovered about the technique analyzed?
- What future research do you recommend?

- Introduction