Here's a list of questions you may want to keep in mind when reading papers:

Context and Problem Statement:

What problems are the authors trying to solve?
Are they important problems? Why or why not?

New Idea:

What new architecture, algorithm, mechanism, methodology, or perspective are the authors proposing?
(How is the new idea different from all other ideas to solve the same problem?)

What to Evaluate?

What, according to the authors, need to be evaluated to confirm the worthiness of their new idea?
Runtime? Throughput? Cache miss ratio? Utilization?

How to Evaluate?

How did the authors go about conducting the evaluation?
Did they prove theorems?
Did they run simulations?
Did they build a system?
Did they collect traces from existing systems?

Was the Evaluation Correct and Adequate?

How was their data collection done?
Do you agree with their analysis of the data?
Do you agree with their conclusions about the data?
Do you have new interpretation of their data?
Can you suggest new ways to evaluate their idea?

Limitations, Drawback, Extension:

Can you think of other aspects of their idea that need to be evaluated?
Can you think of extensions or modifications to their idea to improve it?
How would you evaluate your improvement?
Can you apply their idea or method of evaluation to your own project?