CS885 Fall 2021 - Reinforcement Learning

Paper Critiques

• If you present a paper: one critique of a different paper than the one you will present (10% of final mark)
• If you do not present a paper: three critiques (10% of final mark for each critique)
• To be done individually (i.e., no team)
• Due date: at 11:59 pm on the day before the paper is scheduled for discussion (see schedule)
• Submission: via Crowdmark

Format

• At most 2 pages
• Put yourself in the shoes of a reviewer for a conference or journal
• Give constructive feedback to the authors
• For every criticism, make a suggestion to improve the paper
• Mention also the good aspects of the paper
• If you don't understand something, explain what is unclear about the paper and how clarity could be improved

Suggested questions to structure your critique

Overview

In your opinion:

• What is the paper about?
• What are the contributions?
• What is the take home message?

Significance and originality

In your opinion:

• Are the ideas novel/original?
• Are the contributions significant?

Soundness

In your opinion:

• Are the ideas presented technically correct?
• Are the proofs accurate?
• Are the algorithms correct?

Evaluation

In your opinion:

• Are the ideas/algorithms empirically or theoretically evaluated?
• If some assumptions are made, are they realistic?
• Is scalability demonstrated?

Related work

In your opinion:

• Is the paper properly situated with respect to related work?
• Is there a brief survey of related work?
• Do the authors explain similarities and differences with previous work?

Readability

In your opinion:

• Is the paper well structured?
  • Does the abstract properly and accurately summarize the paper?
  • Do the introduction and conclusion clearly explain the contributions and the take home message?
  • Is the flow of ideas easy to follow?
• Is the paper well written?
  • Are all the technical terms and abbreviations explained?
  • Are there important grammatical errors?
  • Are there a lot of typos?