PRINCETON UNIVERSITY COMPUTER SCIENCE
COS 324: INTRODUCTION TO MACHINE LEARNING
FALL 2020


Administrative Info:
Instructor:
TAs:
Lecture:
Precepts:
Programming Assignments: In Jupyter notebooks (python). Gradual ramp up from zero.

Course Plan

Number
Date
Topic
Reading
Slides/Handouts
Homework
1
9/1 Tue
What is machine learning? Illustration through linear models.


HW0: Getting comfortable with python.
2
9/3 Thu
Linear regression. Intro to basic elements of ML: data, model, sampled training data, algorithm/training, testing (aka generalization). 



3
9/8 Tue
Peek under the hood of the optimization engine. Quick refresher of derivatives and gradients. Gradient descent: the  "natural" algorithm.


Programming HW 1: Linear regression in python
4
9/10 Thu
Modifying learning via changes to cost function. Logistic and multiclass regression. l_2 and l_1 regularizers and their uses in preventing overfitting. (bias towards sparsity).



5
9/15 Tue
 Quick tour of "datascience" through the lens of linear models.  k-fold cross validation. Regression-based analysis in economics, and neuroscience. Interpretability and its pitfalls. (Q&A with guest.)


Programming HW 2: Data science explorations. Effect of regularization.
6
9/17 Thu
Unsupervised learning and its goals. Linear algebra refresher (vectors, inner products, distance). Clustering via k-means.



7
9/22 Tue
Low-dimensional representation  of high dimensional data. Connections to eigenvalues/eigenvectors. ("SVD"); describe but not prove. Eigengenes and Eigenfaces.


Programming HW 3: Clustering and low-dimensional representation
8
9/24 Thu
Refresher of elementary probability (coin tossing, conditional probability and bayes rule). Simple production models for text. n-grams.



9
9/29 Tue
How to evaluate a probabilistic model of text. Perplexity. Training vs test. (Detecting overfitting.)


Programming HW 4:
n-gram model? matrix factorization? Genomics or neuroscience?
10
10/01 Thu
Word embeddings and their fun properties. Constructing word embeddings from n-gram models. (matrix factorization)



11
10/06 Tue
Lore of Gaussians. Understanding sampling and generalization a bit better. (optional; may skip if running late) The nonreproducibility problem in experimental social science.



12
10/8 Thu
Midterm review

Take-home midterm (timed)
13
10/15 Thu
Deep nets. Loss function and gradients. Chain rule


14
10/20 Tue
Backpropagation algorithm.

Programming HW 5: Implementing a simple deep net.
15
10/22 Thu
Convolutional nets and computer vision.


16
10/27 Tue
Convnets contd

Programming HW 6: ConvNets for vision.
17
10/29 Thu
Recurrent nets and applications.(optional; can skip if running late)



18
11/03 Tue
Decision-making under uncertainty: basic elements. (Rational choice framework; expected utility/cost)

No programming HW.
19
11/05 Thu
Markov Decision Processes (MDP) as a model for decision-making in a stochastic environment.Optimum policy



20
11/10 Tue
 Computing optimum policy. (Policy gradient) Guest minilecture by Elad Hazan on linear control and ventilators.


Programming HW7: Computing and using an optimum policy
21
11/12 Thu
(Guest lecture) Game-playing and Alpha Go. 


22
11/17 Tue
Ethics in ML. Explainability, bias, transparency, proper data collection.  (Guest lecturer: ? Maybe Arvind)



23
11/19 Thu
(Guest lecture) Deep Learning and Language Models


24
11/24
Ask me anything.  (Finals review during reading period.)