CS 854 Advanced Topics in Computer Systems: Experimental Performance Evaluation
Please follow the "Course Readings and Course Schedule"
link to see the assignment information.
People trying to enroll in this course may notice that enrollment is closed.
This is because the enrollment limit has been reached.
There is a chance that people who have enrolled won't actually be taking
the course for credit and I MAY be able to allow one or two extra students
(but no promises). So if you want to take the course for credit
please show up to the lectures and we'll try
to sort things out.
Instructor: Tim Brecht
email: email@example.com (include CS854 in the Subject line !!!)
Location: DC 3313
Office Hour: DC 3508, Day and Time: TBA
15% Class Participation
25% Paper Summaries / Presentations
50% Class Project Paper (likely 6-8 pages)
10% Paper/project reviews.
Class Participation is an important
component of this class. You are expected
to have read the papers and be prepared to discuss
them in class.
About the Course
Objectives : An in-depth study of issues, challenges, techniques, and pitfalls in conducting
experimental performance evaluations in computer systems research.
The course will focus on conducting scientifically sound
experimental performance evaluations and drawing valid
conclusions from the results.
The goal is to help students to critically evaluate performance experiments in
existing papers and to design and carry out experimental performance evaluations
in their own work.
Course Meetings : Most likely weekly discussion of 2-3 papers but this format may vary.
Some meetings will involve reading and critiquing the experimental performance evaluation
performed in the papers read that week.
We may perform some case studies, especially if students in the course
are planning on conducting performance evaluations for papers or thesis work.
This would involve a short presentation that describes a problem being studied
and the methodology being used to evaluate the performance and the expected
outcomes. The class will then evaluate and discuss the methodology.
Some of the content may be dictated by the interests of
MORE TO BE ADDED
These are not strictly required but many of the papers and
discussion are likely to revolve around computer systems research
so a good background in the following areas are highly recommended.
- operating systems
- possibly some computer architecture
- systems programming skills
- good communication skills
Course Readings and Course Schedule
Some Resources Used or Talked about In Class
Course Project and Paper
Note that students are not generally permitted to submit the same
work for credit in multiple classes. For example, if a student
has reviewed or presented one of the papers in another seminar
class, he or she should avoid reviewing or presenting it again for
The general university policy:
- Academic Integrity: In order to maintain a culture of
academic integrity, members of the University of Waterloo
community are expected to promote honesty, trust, fairness,
respect and responsibility. [Check http://www.uwaterloo.ca/academicintegrity/
for more information.]
- Grievance: A student who believes that a decision
affecting some aspect of his/her university life has been unfair
or unreasonable may have grounds for initiating a grievance.
Read Policy 70, Student Petitions and Grievances, Section 4, http://www.adm.uwaterloo.ca/infosec/Policies/policy70.htm.
When in doubt please be certain to contact the department's
administrative assistant who will provide further
- Discipline: A student is expected to know what
constitutes academic integrity [check http://www.uwaterloo.ca/academicintegrity/]
to avoid committing an academic offence, and to take
responsibility for his/her actions. A student who is unsure
whether an action constitutes an offence, or who needs help in
learning how to avoid offences (e.g., plagiarism, cheating) or
about "rules" for group work/collaboration should seek guidance
from the course instructor, academic advisor, or the
undergraduate Associate Dean. For information on categories of
offences and types of penalties, students should refer to Policy
71, Student Discipline, http://www.adm.uwaterloo.ca/infosec/Policies/policy71.htm.
For typical penalties check Guidelines for the Assessment of
- Appeals: A decision made or penalty imposed under
Policy 70 (Student Petitions and Grievances) (other than a
petition) or Policy 71 (Student Discipline) may be appealed if
there is a ground. A student who believes he/she has a ground
for an appeal should refer to Policy 72 (Student Appeals) http://www.adm.uwaterloo.ca/infosec/Policies/policy72.htm.
Note for Students with Disabilities
The Office for Persons with Disabilities (OPD), located in
Needles Hall, Room 1132, collaborates with all academic departments
to arrange appropriate accommodations for students with disabilities
without compromising the academic integrity of the curriculum. If
you require academic accommodations to lessen the impact of your
disability, please register with the OPD at the beginning of each