CS480/680 Winter 2023 - Introduction to Machine Learning

Overview

Graduate students enrolled in CS680 only
To be done individually (i.e., no teams)
To search for papers on a topic, use Google scholar. Most papers available on the web can be found via Google scholar. Furthermore, most links in Google scholar are free. As a student you should never have to pay to download a paper. If Google scholar does not provide a free link, you should be able to access the paper freely via the University's internet or the University's library website.

Options

Option A (Literature survey):
- Pick a problem that interests you
- Search the literature for machine learning approaches to tackle this problem
- Survey and discuss the relative strengths of each approach
- If you'd like to see examples of survey papers in AI, have a look at the IJCAI-2021 survey track
Option B (Empirical evaluation):
- Pick a problem that interests you.
- Implement and experiment with several machine learning techniques to tackle this problem.
Option C (Algorithm design):
- Identify a problem for which there are no satisfying approaches.
- Develop a new machine learning technique to tackle this problem.
- Analyze theoretically and/or empirically the performance of your technique.
Option D (Dataset/Simulator/Benchmark design):
- Identify a problem for which there is a lack of datasets or benchmarks to evaluate machine learning algorithms.
- Collect a dataset or design a new benchmark to evaluate machine learning algorithms.
- Demonstrate how some baseline machine learning algorithms perform with your dataset or benchmark.
- If you'd like to see examples of papers describing datasets or benchmarks, have a look at the NeurIPS-2021 datasets and benchmarks track
Option E (Theoretical analysis):
- Identify a problem or machine learning technique for which the properties (e.g., complexity, performance) are not well understood.
- Analyze the properties of this problem or technique.

Proposal (no mark)

Submit electronically on the LEARN website by March 8 (11:59 pm)
At most one page (excluding references)
Use the JMLR format: https://www.jmlr.org/format/format.html
Which option did you pick?
Option A (Literature survey):

What is the problem?
Cite 8 to 12 papers that you plan to survey.

Option B (Empirical evaluation):
- What is the problem?
- What machine learning techniques do you plan to experiment with?
- Cite 4 to 8 related papers that you plan to review.
Option C (Algorithm design):
- What is the problem?
- Why are there no satisfying approaches?
- What is the intuition behind the new technique that you plan to develop?
- Cite 4 to 8 related papers that you plan to review.
Option D (Dataset/Simulator/Benchmark design):
- What is the machine learning problem for which there is a lack of datasets or benchmarks?
- What dataset or benchmark do you plan to design?
- Cite 4 to 8 related papers that you plan to review.
Option E (Theoretical analysis):
- What is the problem or technique that you plan to analyze?
- What properties would you like to analyze/prove about this problem or technique?
- Cite 4 to 8 related papers that you plan to review.

Report (25% of final mark)

At most 8 pages (excluding references)
Use the JMLR format: https://www.jmlr.org/format/format.html
Explain the big picture and any necessary detail
Submit electronically on the LEARN website by April 17 (11:59 pm)

Suggested Structure for the Report

Option A (Literature survey):
- Introduction
  - What is the problem?
  - Why is it an important problem?
- Survey
  - Summarize the range of techniques by highlighting their strengths and weaknesses (i.e., the 8-12 papers that you read)
  - Tip: this summary should not be a laundry list of techniques with an independent paragraph for each technique
  - Suggestion: organize your summary based on desirable properties of the techniques
- Analysis:
  - What is the state of the art?
  - Any open problem?
- Conclusion
  - What have you learned?
  - What future research do you recommend?
Option B (Empirical evaluation):
- Introduction
  - What is the problem?
  - Why is it an important problem?
- Techniques to tackle the problem
  - Brief review of previous work concerning this problem (i.e., the 4-8 papers that you read)
  - Brief description of the techniques chosen and why
- Empirical evaluation
  - Compare empirically the techniques for complexity, performance, ease of use, etc.
- Conclusion:
  - What is the best technique?
  - Is any technique good enough to declare the problem solved?
  - What future research do you recommend?
Option C (Algorithm design):
- Introduction
  - What is the problem?
  - Why can't any of the existing techniques effectively tackle this problem?
  - What is the intuition behind the technique that you have developed?
- Techniques to tackle the problem
  - Brief review of previous work concerning this problem (i.e., the 4-8 papers that you read)
  - Describe the technique that you developed
  - Brief description of the existing techniques that you will compare to
- Evaluation
  - Analyze and compare (empirically or theoretically) your new approach to existing approaches
- Conclusion:
  - Can your new technique effectively tackle the problem?
  - What future research do you recommend?
Option D (Dataset/Benchmark):
- Introduction
  - What is the problem?
  - Why aren't existing datasets or benchmarks sufficient to evaluate ML techniques for this problem?
- Proposed Dataset or Benchmark
  - Describe the proposed dataset or benchmnark
  - Describe the properties of the dataset or benchmark that are unique
- Evaluation
  - Brief description of some baseline ML techniques that you plan to evaluate
  - Compare empirically the baseline ML techniques with your dataset or benchmark
- Conclusion:
  - What are the most important weaknesses of existing baselines that your dataset or benchmark highlighted
  - What future research do you recommend?
Option E (Theoretical analysis):
- Introduction
  - What is the problem or technique?
  - What properties did you analyze/prove about this problem or technique?
- Analysis
  - Brief survey of previous work concerning this problem (i.e., the 4-8 papers that you read)
  - Describe the analysis performed
- Conclusion:
  - What have you discovered about the technique analyzed?
  - What future research do you recommend?