This course is an elementary introduction to a machine learning technique called deep learning (also called deep neural nets), as well as its applications to a variety of domains, including image classification, speech recognition, and natural language processing. Along the way the course also provides an intuitive introduction to basic notions such as supervised vs unsupervised learning, linear and logistic regression, continuous optimization (especially variants of gradient descent), generalization theory and overfitting, regularizers, and probabilistic modeling. The homeworks explore key concepts and simple applications, and the final project allows an in-depth exploration of a particular application area.
Instructor: Yingyu Liang, CS building 103b, Reception hours: Thu 3:00-4:00
Teaching Assistant: Bochao Wang, Electrical Engineering, C319B, Reception hours: Mon + Tue, 11:00-12:00
COS 126 General Computer Science (or equivalent) and COS 340 Reasoning About Computation.
Mostly need linear algebra, calculus, probability, and some programming knowledge.
Attendance and the use of electronic devices: Attendance is expected at all lectures. The use of laptops and similar devices for note-taking is permitted.
|#||Date||Topic||Lecture notes||Extra reading||Problem sets|
|1||02/01||Motivation and logistics of the course||slides||Nature review on deep learning|
|2||02/03||Machine learning basics 1: linear regression||slides||Math background: Chapter 2-4 of the textbook|
|3||02/08||Machine learning basics 2: linear classification||slides|
|4||02/10||Machine learning basics 3: Perceptron and SGD||slides||Chapter 4.1.7 and 4.2 in Pattern Recognition and Machine Learning||homework 1|
|5||02/15||Machine learning basics 4: SVM I||slides||Andrew Ng's note on SVM|
|6||02/18||Machine learning basics 5: SVM II||slides||Appendix B (Convex Optimization) in Foundations of Machine Learning|
|7||02/22||Machine learning basics 6: overfitting||slides||Chapter 5.1-5.4 of the textbook|
|8||02/24||Machine learning basics 7: multiclass classification||slides||homework 2|
|9||02/29||Deep learning 1: feedforward neural networks||slides||Chapter 6 of the textbook|
|10||03/02||Deep learning 2: backpropagation||slides|
|11||03/07||Deep learning 3: regularization I||slides||Chapter 7.1-7.3 of the textbook|
|12||03/09||Deep learning 4: regularization II||slides||Paper on dropout regularization||homework 3|
|13||03/21||Deep learning 5: convolution||slides||Chapter 9.6-9.10 of the textbook|
|14||03/23||Deep learning 6: convolutional neural networks||slides||Chapter 8.3-8.4 of the textbook|
|15||03/28||Deep learning 7: factor analysis||slides||Chapter 13 of the textbook|
|16||03/30||Deep learning 8: autoencoder and DBM||slides||Chapter 14.3-14.5 of the textbook||homework 4|
|17||04/04||Deep learning 9: recurrent neural networks||slides||Chapter 10.5-10.7 of the textbook|
|18||04/11||Deep learning 10: natural language processing||slides|
|19||04/13||Guest lecture by Tengyu Ma: word embedding theory||slides||Paper on arXiv|
|20||04/18||Deep learning 11: practical methodology||slides|| Chapter 11 of the textbook;
Practical Recommendations for Gradient-Based Training of Deep Architectures from Yoshua Bengio