CS 854 Advanced Topics in Computer Systems: Experimental Performance Evaluation (Fall 2018)

UNDER CONSTRUCTION: Many of the links below will not work yet.

NOTE: The first class is on Friday September 7, 2018 from 10:00 am - 12:20 pm.
Please do not skip this class to attend the information session about graduate courses.
I will provide information and background for the course in this first class.
The information sessions will be recorded and you can view the videos later.

Course Overview Video

Link to video of me explaining what the course is about (from 2017)
The course will be quite similar in 2018

Course Info

Day: Friday
Time: 10:00-12:20
Location: DC 2568
URL : http://cs.uwaterloo.ca/~brecht/courses/854/
Piazza piazza.com/uwaterloo.ca/fall2018/cs854

Instructor Info

Instructor: Tim Brecht
email: brecht@cs.uwaterloo.ca (include CS854 in the Subject line !!!)
Office Hours: TBA
Office Location: DC 3508
Home Page: http://cs.uwaterloo.ca/~brecht

Grading (tentative)

20% Class Participation
20% Paper Summaries / Presentations
50% Class Project Paper (likely 6-7 pages)
10% Paper/project reviews.

Class Participation

Class Participation is an important component of this class. You are expected to have read the papers and be prepared to discuss them in class.

About the Course

Objectives : An in-depth study of issues, challenges, techniques, and pitfalls associated with conducting experimental performance evaluations. The course will focus on conducting scientifically sound experimental performance evaluations and drawing valid conclusions from the results. The goal is to help students to critically evaluate performance experiments in existing papers and to design and carry out experimental performance evaluations in their own work.
Course Meetings : Mostly weekly discussion of 2-3 papers but this format may vary. Some meetings will involve reading and critiquing the experimental performance evaluation performed in the papers read that week. We may perform some case studies, especially if students in the course are planning on conducting performance evaluations for papers or thesis work. This would involve a short presentation that describes a problem being studied and the methodology being used to evaluate the performance and the expected outcomes. The class will then evaluate and discuss the methodology.
Content : Some of the content may be dictated by the interests of the students.
MORE TO BE ADDED

Expected Background

These are not strictly required but many of the papers and discussion are likely to revolve around computer systems research so a good background in the following areas are recommended.

statistics
systems programming skills
good communication skills
operating systems
networking
possibly some computer architecture

Course Readings and Course Schedule

Hopefully some of this will be decided in conjunction with the class participants.
Possible Course Readings and Course Schedule

Course Project and Paper

It will be possible to work on your own or in small groups of 2 or possibly 3.
The project paper will be due on the last day of classes.
After all projects are handed each student will read and provide a conference style review of some subset of the papers. This will be followed by a meeting that takes the form of a program committee meeting.
All projects and groups must be approved.
Information about what I'd like you to talk about for the project proposal.
WORKING DOCUMENT : The Final Project / Paper Logistics (THIS WILL CHANGE)

Academic Integrity

Note that students are not generally permitted to submit the same work for credit in multiple classes. For example, if a student has reviewed or presented one of the papers in another seminar class, he or she should avoid reviewing or presenting it again for this class.

The general university policy:

Academic Integrity: In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility. [Check http://www.uwaterloo.ca/academicintegrity/ for more information.]
Grievance: A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70, Student Petitions and Grievances, Section 4, http://www.adm.uwaterloo.ca/infosec/Policies/policy70.htm. When in doubt please be certain to contact the department's administrative assistant who will provide further assistance.
Discipline: A student is expected to know what constitutes academic integrity [check http://www.uwaterloo.ca/academicintegrity/] to avoid committing an academic offence, and to take responsibility for his/her actions. A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) or about "rules" for group work/collaboration should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean. For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline, http://www.adm.uwaterloo.ca/infosec/Policies/policy71.htm. For typical penalties check Guidelines for the Assessment of Penalties, http://www.adm.uwaterloo.ca/infosec/guidelines/penaltyguidelines.htm.
Appeals: A decision made or penalty imposed under Policy 70 (Student Petitions and Grievances) (other than a petition) or Policy 71 (Student Discipline) may be appealed if there is a ground. A student who believes he/she has a ground for an appeal should refer to Policy 72 (Student Appeals) http://www.adm.uwaterloo.ca/infosec/Policies/policy72.htm.

Note for Students with Disabilities

The Office for Persons with Disabilities (OPD), located in Needles Hall, Room 1132, collaborates with all academic departments to arrange appropriate accommodations for students with disabilities without compromising the academic integrity of the curriculum. If you require academic accommodations to lessen the impact of your disability, please register with the OPD at the beginning of each academic term.