||What is machine learning? Illustration
through linear models.
||HW0: Getting comfortable with python.
||Linear regression. Intro to basic elements of
ML: data, model, sampled training data, algorithm/training,
testing (aka generalization).
||Peek under the hood of the optimization
engine. Quick refresher of derivatives and gradients.
Gradient descent: the "natural" algorithm.
||Programming HW 1: Linear regression in python
||Modifying learning via changes to cost
function. Logistic and multiclass regression. l_2 and l_1
regularizers and their uses in preventing overfitting. (bias
|| Quick tour of "datascience" through the
lens of linear models. k-fold cross validation.
Regression-based analysis in economics, and neuroscience.
Interpretability and its pitfalls. (Q&A with guest.)
||Programming HW 2: Data science explorations.
Effect of regularization.
||Unsupervised learning and its goals. Linear
algebra refresher (vectors, inner products, distance).
Clustering via k-means.
||Low-dimensional representation of high
dimensional data. Connections to eigenvalues/eigenvectors.
("SVD"); describe but not prove. Eigengenes and Eigenfaces.
||Programming HW 3: Clustering and
||Refresher of elementary probability (coin
tossing, conditional probability and bayes rule). Simple
production models for text. n-grams.
||How to evaluate a probabilistic model of
text. Perplexity. Training vs test. (Detecting overfitting.)
||Programming HW 4:
n-gram model? matrix factorization? Genomics or neuroscience?
||Word embeddings and their fun properties.
Constructing word embeddings from n-gram models. (matrix
||Lore of Gaussians. Understanding sampling and
generalization a bit better. (optional; may skip if running
late) The nonreproducibility problem in experimental social
||Midterm review||Take-home midterm (timed)|
||Deep nets. Loss function and gradients. Chain rule|
||Backpropagation algorithm.||Programming HW 5: Implementing a simple deep net.|
||Convolutional nets and computer vision.|
||Convnets contd||Programming HW 6: ConvNets for vision.|
||Recurrent nets and applications.(optional;
can skip if running late)
||Decision-making under uncertainty: basic elements. (Rational choice framework; expected utility/cost)||No programming HW.
||Markov Decision Processes (MDP) as a model
for decision-making in a stochastic environment.Optimum
|| Computing optimum policy. (Policy
gradient) Guest minilecture by Elad Hazan on linear control
||Programming HW7: Computing and using an optimum policy|
||(Guest lecture) Game-playing and Alpha Go.|
||Ethics in ML. Explainability, bias,
transparency, proper data collection. (Guest lecturer:
? Maybe Arvind)
||(Guest lecture) Deep Learning and Language Models|
||Ask me anything. (Finals review during