CS480/680 Spring 2019 - Introduction to Machine Learning

This is a tentative schedule only. As the course progresses, the schedule will be adjusted.

[GBC] Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning (2016) freely available online
[HTF] Trevor Hastie, Robert Tibshirani and Jerome Friedman, Elements of Statistical Learning (2nd edition, 2009) freely available online
[D] Hal Daume III, A Course in Machine Learning (2017) freely available online
[B] Christopher Bishop, Pattern Recognition and Machine Learning (2006) freely available online
[MRT] Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar, Foundations of Machine Learning (2012) freely available online
[SSBD] Shai Shalev-Shwartz, Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms (2014) freely available online
[RN] Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach (3rd Edition) (2010)
[M] Kevin Murphy, Machine Learning: A Probabilistic Perspective (2012)

Lecture	Date	Topic	Readings (textbooks)
1	May 6	Introduction to Machine Learning (Lecture slides) (video)
2	May 8	K-nearest neighbours (Lecture slides (slide 20 revised on Jan 11)) (video)	[RN] Sec. 18.8.1, [HTF] Sec. 2.3.2, [D] Chapt. 3, [B] Sec. 2.5.2, [M] Sec. 1.4.2
3	May 13	Linear regression (Lecture slides) (Notation Reference Sheet) (video)	[RN] Sec. 18.6.1, [HTF] Sec. 2.3.1, [D] Sec. 7.6, [B] Sec. 3.1, [M] Sec. 1.4.5
4	May 15	Statistical learning (Lecture slides) (video)	[RN] Sec. 20.1, 20.2, [M] Sec. 2.2, 3.2
	May 20	No Lecture (Victoria Day)
5	May 22	Linear regression by maximum likelihood, maximum a posteriori, Bayesian learning (Lecture slides (Slide 14 revised on June 24)) (video)	[B] Sec. 3.1-3.3, [M] Chap. 7
	May 24	Assignment 1 due (11:59 pm)
6	May 27	Project ideas: Tools for surveys (Paulo Pacheco): slides video Kaggle datasets and competitions (Mike Rudd): slides video Normalizing flows (Priyank Jaini): video Unsupervised word translation (Kira Selby): slides video Fact checking and reinforcement learning (Vik Goel): slides video Sum-product networks (Pranav Subramani): slides video EM and mixture models (Guojun Zhang): slides video Model compression for NLP (Ashutosh Adhikari): slides video
7	May 29	Mixture of Gaussians (Lecture slides) (video)	[B] Sec. 4.2, [M] Sec. 4.2
8	Jun 3	Logistic regression, generalized linear models (Lecture slides (Slide 11 revised on June 20, Slide 18 revised on June 26)) (video)	[RN] Sec. 18.6.4, [B] Sec. 4.3, [M] Chap. 8, [HTF] Sec. 4.4
9	Jun 5	Perceptrons, single layer neural networks (Lecture slides)(video)	[D] Chapt. 4, [HTF] Chapt. 11, [B] Sec. 4.1.7, 5.1, [M] Sec. 8.5.4, [RN] Sec. 18.7
	Jun 10	Assignment 2 due (11:59 pm)
10	Jun 10	Multi-layer neural networks, backpropagation (Lecture slides) (video)	[D] Chapt. 10, [HTF] Chapt. 11, [B] Sec. 5.2, 5.3, [M] Sec. 16.5, [RN] Sec. 18.7
11	Jun 12	Kernel methods (Lecture slides) (video)	[D] Chapt. 11, [B] Sec. 6.1, 6.2 [M] Sec. 14.1, 14.2 [HTF] Chap. 6
	Jun 14	Project proposal due (11:59 pm)
12	Jun 17	Gaussian Processes (Lecture slides (slide 11 revised on June 24)) (video)	[B] Sec. 6.4 [M] Chap. 15 [HTF] Sec. 8.3
13	Jun 19	Support vector machines (Lecture slides (Slide 20 revised on June 24)) (video)	[B] Sec. 7.1 [D] Sec. 11.5-11.6 [HTF] Chap. 12 [M] Sec. 14.5 [RN] Sec. 18.9 [MRT] Chap. 4
	Jun 21	Midterm (8:30 pm - 9:50 pm) STC 0010: Students whose last name starts with A-G STC 1012: Students whose last name starts with H-Z
14	Jun 24	Support vector machines continued (Lecture slides) (video)	[B] Sec. 7.1 [D] Sec. 6.7 [HTF] Chap. 12 [M] Sec. 14.5 [RN] Sec. 18.9 [MRT] Chap. 4
15	Jun 26	Deep neural networks (Lecture slides) (video)	[GBC] Chap. 6, 7, 8
	Jun 30	Assignment 3 due (11:59 pm)
	July 1	No Lecture (Canada Day) (The lecture is moved to Tuesday July 2, exceptionally)
16	Jul 2	Convolutional neural networks (Lecture slides (slide 6 revised July 10)) (video)	[GBC] Chap. 9
17	Jul 3	Hidden Markov models (Lecture slides (Slide 16 revised August 12)) (video)	[RN] Sec. 15.3 [B] Sec. 13.1-13.2 [M] Sec. 17.3-17.5
18	Jul 8	Recurrent neural networks (Lecture slides (slide 13 revised July 13)) (video)	[GBC] Chap. 10
19	Jul 10	Attention and transformer networks (Lecture slides (slides 5, 6, 8, 9 revised July 13, 15, 30)) (video)	[Vaswani et al., Attention is All You Need, NeurIPS, 2017]
	Jul 12	Assignment 4 due (11:59 pm)
20	Jul 15	Autoencoders (Lecture slides) (video)	[GBC] Chap. 14
21	Jul 17	Generative networks (variational autoencoders and generative adversarial networks) (Lecture slides) (video)	[GBC] Chap. 20
22	Jul 22	Ensemble learning: bagging and boosting (Lecture slides) (video)	[RN] Sec 18.10, [M] Sec. 16.2.5, [B] Chap. 14, [HTF] Chap. 15-16, [D] Chap. 11
23	Jul 24	Normalizing Flows (guest lecture by Priyank Jaini) (Lecture slides) (video)
24	Jul 29	Gradient boosting, bagging, decision forests (Lecture slides (Slide 5 revised August 13)) (video)	[RN] Sec 18.10, [M] Sec. 16.2.5, 16.4.5, [B] Chap. 14, [HTF] Chap. 10, 15-16, [D] Chap. 13
	Jul 30	Assignment 5 due (11:59 pm)
	Aug 9	Project report due (11:59 pm)