CS 846 : Recommendation Systems for Software Engineering

Recommendation systems are frequently used to derive contextualized recommendations for a variety of tasks. Given the wide variety of structured artifacts generated while generating software, recommendation systems have the opportunity to help developers to complete their tasks. In this course, we will examine software recommendation systems, with a particular focus on the approaches for generating recommendations. The course will be seminar-based and will involve weekly reading and discussion. The project component will involve building and evaluating a recommendation system for software engineers.

The course will focus primarily on techniques used to generate recommendations for software engineers. Two side topics (recommendation delivery and evaluating RSSEs) will also be explored. The high-level themes for the course are:

• Recommendation Delivery
• Evaluating RSSEs
• Heuristics
• Data Mining
• Supervised Machine Learning
• Unsupervised Machine Learning
• Formal Concept Analysis
• Data Integration

This course is offered by the David R. Cheriton School of Computer Science at the University of Waterloo.

Lectures are held Wednesday afternoons from 13:00 to 15:50 in DC 3313. My office hours are TBD but will be held in DC 3351. Official scheduling details can be found in the administrative entry.

Important dates and information will be posted to @cs846.

The best way to get in touch with me is via email, or in class.

The course schedule will be entirely tracked in the Online Spreadsheet. Classes will start Jan 8 and continue until April 2, with the exception of Feb 19 (Reading week).

This will be a paper-based seminar course. Each week we will read and discuss three papers associated with a particular recommendation system theme. It is expected that everyone will read all three papers and be prepared to discuss them. The first paper will not typically be a software engineering paper but will be 'seminal' from the research area associated with the theme. The goal of presenting this paper is to provide a broad overview of how the approach works and what its strengths and weaknesses are in the domaine of RSSEs. This presentation can include a mini-tutorial that clearly describes the approach and helps the class to understand how it functions. An overview of how the approach has evolved since it was first presented might also be appropriate.

The first of the two accompanying papers will present an RSSE that used the approach described in the seminal paper to accomplish a specific software engineering task. The second accompanying paper will tackle the same (or tangentially similar) task as the previous RSSE paper, but specifically without using the same approach.

When selecting papers you can also choose a fourth reference paper/set of slides/other material for additional reading. This resource is not required reading, but may be of interest to the class.

Two students will run each week's seminar, presenting the three papers and leading the discussion. It is expected that the presentations for the papers will take 45-60 minutes and the discussion will last for an additional 90-120 minutes.

While reading the papers you should be able to answer the following five questions:

1. What were the primary contributions of the paper as the author sees it?
2. What were the main contributions of the paper as you (the reader) see it?
3. How does this work move the research forward (or how does the work apply to you)?
4. How could this research be extended?
5. How could this research be applied in practice?

While presenters should keep these questions in mind, the audience in particular should think about them specifically while they are reading the paper.

It will be up to you to select the theme you wish to present. To sign up, enter your name in the presenter list in the course schedule sheet in one of the available (non-blacked out) boxes. You will also be responsible for selecting the papers you will discuss; the paper list must be finalized by Jan 22. I have pre-assigned the papers for Jan 15 and Jan 22 to encourage people to sign up for those days. I have compiled a list of potential RSSE papers, although you should not feel constrained by this list; if you find others feel free to let me know and I can add them. Once you have selected your papers, please send me email so I can ensure they are appropriate.

If you are auditing the course, please do not sign up for a topic right away; instead add your name to the auditors list at the bottom of the page. Auditors will be linked with topics after the for-credit students have made their selections.

Grading: The seminar portion of the course is worth 30% of the final course grade. The grade will be assessed based on the quality of the selected papers, the quality of the presentation, and the effectiveness of the discussion.

The project forms an integral part of this course and must be completed in a group of two or three. The project will involve building a recommendation system to solve some specific software engineering information need.

The goal of the project is to identify an information deficiency developers encounter in practice, determine how a recommendation system can alleviate this problem, and validate that the solution helps with the initial problem. I would recommend drawing upon your experience as you write code to identify some problem that has inhibited you in the past and fix it.

The outcome of this project will be a short (5-6 page) paper describing the problem, your solution, a comparison to related approaches, and some form of validation.

There are two deliverables for the project:

1. Project proposal. Before you undertake your project you will need to submit a proposal for approval. The proposal should be short (max 1 page PDF in ACM format). The proposal should include a problem statement, the motivation for the project, and set of objectives you aim to accomplish. I will read these and provide comments. The proposal is not for marks but _must_ be completed in order to pass the course.
   This will be due on January 29 @ 0800 via email.
2. Written report. All reports must be formatted according to the ACM format and submitted as a PDF. This artifact will constitute 100% of the project grade.
   This will be due on March 19 @ 0800 via Easychair.

Grading: The project comprises 50% of your final grade. The assessment will be solely based on the quality of the research paper.

In the last week of the class we will formally review all the papers from the course projects (using standard program committee review forms) and have a PC meeting describing the strengths and weaknesses of each paper. All projects will be submitted via a EasyChair (a conference management system). Each paper will receive at least 8 reviews from class members (I will review every paper). Michael Ernst has compiled a series of informative links for creating effective reviews. Here are links to two different reviews as well (one positive and one negative) if you want concrete examples.

Grading: Your peer-reviews will comprise 10% of your final mark.

For a seminar-based discussion class to function, everyone needs to take part. At the end of the course you will submit a short 'participation diary' in which you will suggest a letter grade for your participation in the course and a justification for this grade. Since there are only a small number of classes, it might make sense to jot down a sentence or two after each class so you can keep track of how you participated in the discussions in the class.

Grading: I will use the participation diary, your suggested grade, and my own notes to assign the final participation mark. This will be worth 10% of your final grade.

Academic Integrity

• In order to maintain a culture of academic integrity, members of the University of Waterloo community are expected to promote honesty, trust, fairness, respect and responsibility. [See the academic integrity site for more information.]

Grievance

• A student who believes that a decision affecting some aspect of his/her university life has been unfair or unreasonable may have grounds for initiating a grievance. Read Policy 70, Student Petitions and Grievances, Section 4.
• When in doubt please be certain to contact the department’s administrative assistant who will provide further assistance.

Discipline

• A student is expected to know what constitutes academic integrity to avoid committing an academic offence, and to take responsibility for his/her actions.
• A student who is unsure whether an action constitutes an offence, or who needs help in learning how to avoid offences (e.g., plagiarism, cheating) or about “rules” for group work/collaboration should seek guidance from the course instructor, academic advisor, or the undergraduate Associate Dean.
• For information on categories of offences and types of penalties, students should refer to Policy 71, Student Discipline.
• For typical penalties check Guidelines for the Assessment of Penalties.

Appeals

• A decision made or penalty imposed under Policy 70 (Student Petitions and Grievances) (other than a petition) or Policy 71 (Student Discipline) may be appealed if there is a ground.
• A student who believes he/she has a ground for an appeal should refer to Policy 72 (Student Appeals).