CS 846-002: Software Analytics for Release Pipelines
In this course, students will learn about the application of analytics (i.e., the discovery and communication of meaningful patterns in data) to software release processes. Throughout the course, students will:
- study, discuss, and critique research papers appearing in the current literature
- report on an original project or a replication of a paper in the area
The main objective of this course is
to introduce students to how the field of software analytics (i.e., the discovery and communication of meaningful patterns in software data) can be leveraged to improve software release pipelines (i.e., the process by which new software releases are produced). By the end of this course, students should understand:
- the raw data that is produced by modern software release and operations practices
- how to architect, design, and implement a software analytics system, which processes the raw data to derive value
- the measures by which a software analytics system may be evaluated
The course is comprised of three modules. In addition to laying out course expectations, the instructor-lead preliminaries module introduces students to (a) the basics of software analytics and (b) the sources of data that one should expect in modern software release and operations processes. In the integration practices and release practices modules, students will be exposed to key findings and recent trends in the literature.
Students will be evaluated using the following breakdown:
Adversarial Presentations (10%)
Each scheduled paper will be assigned to two students. One student will present the paper and champion it in a discussion period (see Champion Presentation below). The other student will take an adversarial position, preparing a 10-minute presentation that points out controversial points in the paper and limitations in its scope, evaluation, or applicability.
Each student will play the role of adversary at least once, with the presentation being worth 10% of the final grade. If a student is asked to present more than once, the highest grade will be retained and the lowest grade will be dropped.
Champion Presentations (10%)
Each paper will be assigned a champion. The champion will deliver a 15-minute presentation on the paper to the class. The champion will also lead break-out group discussions where other students may ask detailed questions about the work.
Each student will play the role of champion at least once. Again, if a student is asked to present more than once, the lowest grade will be dropped and the remaining grade will be worth 10% of the final grade.
Students must submit a 1,000-word maximum (strict!) critique for four papers in the schedule. Students may not submit a critique for papers for which they are scheduled to act as the champion or the adversary. Your critiques must follow the template for critiques.
Each critique will be worth 5% of the final grade. For students who submit more than four critiques, their highest four grades will be retained and their lower grades will be dropped.
Each student will produce a individual project. The project may be either a replication of one of the papers scheduled for this course or an original project of an agreed upon topic and scope. Regardless of which project option is selected, each student should schedule a brief meeting with the instructor to discuss the project plan.
The grade for the project is further decomposed into:
- Proposal (5%): The proposal (one page, IEEE format) should provide a brief motivation of the problem, a detailed discussion of the data that will be used, a timeline of project milestones, and the expected outcome.
- Final presentation (15%): Each project will be presented and discussed during the final two weeks of the semester.
- Final report (30%): The final project report (five pages, IEEE format) will argue for the importance of the project, situate the project with respect to the related work, describe the design and results of the analysis, and discuss the broader implications of the results.
Being a seminar-style graduate course, the value that students derived from the material will be proportional to the effort they put into engaging with the content both in preperation for and during class. Students are expected to read all of the scheduled papers, and come to class prepared to take part in the discussion.