Guidelines for Paper Critiques

Read the assigned paper and complete a 2000 words review summarizing the paper's acceptability to the "CS 755 Selected Papers" series. You should address the following points (weighted as indicated). Use your own words and, except for the summary in the first point, do not merely rephrase ideas from the paper.

Your summary should be submitted as a PDF file, in single-column format. Use at least a 10 point font.

  1. Briefly summarize the paper. (25%)

    This part should be self-contained, i.e., it should be possible to understand the summary without having read the full paper. Your target audience is a (hypothetical) CS 755 student who has not read the paper. In other words, assume your reader already has some famiarity with the topics covered in this course, but not with the material in the paper you are summarizing.

  2. Describe the importance of the problem. (10%)

    Consider, for example, the following questions: Is it a new, significant, and relevant problem? Do the authors sufficiently motivate the problem? Did you learn anything? Does the problem fit the scope of the course?

  3. Describe the quality of the presentation. (10%)

    Consider, for example, the following questions: Do the authors cite the relevant literature? Do the authors clearly describe what was done and/or how it was studied? How well do the title, abstract, and summary convey the intent of the paper? Is the writing clear and concise? Do the authors provide the right level of detail? Are the figures informative? Are the mechanics (e.g., English usage) adequate?

  4. Evaluate the papers' contributions. (55%)
    1. Describe the quality of the research.
      • Enumerate three (3) of the paper's strongest points. For each one, indicate whether this is a major or minor strength. Explain your reasoning.
      • Enumerate three (3) of the paper's weakest points. For each one, indicate whether this is a major or minor weakness and elaborate on how the weakness could be corrected or eliminated.

      In addressing this part, consider, for example, the following aspects: Are the results described in the paper correct? Does the analysis use appropriate methods? Does the analysis cover all the important issues? Is the argument logical? How much has been implemented? Do the authors provide sufficient data and/or well-supported arguments?

      Please do not use presentation as one of the strong/weak points in this section -- I don't want to see "presentation was bad" or "presentation was good" comments -- you address this under point 3.

    2. Describe two or three (2-3) of your own additional ideas for possible extensions to the work.

      You may wish to consider interesting short-term research or development that could be conducted within 6 months to explore the ideas further or help validate or repudiate the results. You could also consider long-term research or development that could be initiated to contribute more substantially to the ideas proposed in the paper.

      (Note that this part is especially important if you did not identify major weaknesses in the work. Explanations of how to overcome any of the identified weaknesses should not be repeated. If you choose to mention any of the extensions suggested in the paper, you must elaborate the ideas significantly beyond what is suggested.)

  5. Provide a numerical rating of the paper's acceptability. (0 marks)
    1 = excellent 2 = good 3 = fair 4 = neutral 5 = poor
Note: You will likely find Keshav's essay entitled How to Read a Paper instructive.

Another good source for public reviews of computing literature can be found in the ACM Computing Reviews.

These guidelines are based on Martin Karsten's guidelines that can be found here.