COS 302 / SML 305: Mathematics for Numerical Computing and Machine Learning

Course Logistics

• [required] Lecture Video: Introduction and Logistics (18:37)
• [required] Reading: Course Syllabus

Vectors

• [required] Lecture Video: Vector Basics (15:03)
• [required] Reading: MML 2.0
• [optional] Video: 3Blue1Brown on vectors

Matrices

• [required] Lecture Video: Matrix Basics (18:44)
• [required] Reading: MML 2.2
• [optional] Video: 3Blue1Brown on matrix multiplication

Systems of Linear Equations

• [required] Lecture Video: Solving Linear Systems (36:44)
• [required] Reading: MML 2.1
• [required] Reading: MML 2.3

Matrix Inversion

• [required] Lecture Video: Matrix Inversion (25:06)
• [optional] Blog post: Don't invert that matrix.

Vector Spaces

• [required] Lecture Video: Vector Spaces (16:11)
• [required] MML 2.4

Linear Independence, Basis and Rank

• [required] Lecture Video: Linear Independence, Basis, and Rank (14:00)
• [required] MML 2.5-2.6
• [optional] 3Blue1Brown video on basis vectors

Linear Maps

• [required] Lecture Video: Linear maps (9:35)
• [required] MML 2.7-2.8
• [optional] 3Blue1Brown video on linear transformations and matrices

Change of Basis

• [required] Lecture Video: Change of Basis (11:00)
• [required] MML 2.7-2.8
• [optional] 3Blue1Brown video on change of basis

Norms and Inner Products

• [required] Lecture Video: Norms and Inner Products (18:58)
• [required] MML 3.0-3.3

Orthogonality

• [required] Lecture Video: Orthogonality and Projection (17:16)
• [required] MML 3.4-3.8

Gram-Schmidt Orthogonalization

• [required] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
• [required] MML 3.4-3.8
• [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

Determinant and Trace

• [required] Lecture Video: Matrix Invariants (12:28)
• [required] MML 4.0-4.1
• [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
• [optional] 3Blue1Brown video on determinants

Eigenvectors and Eigenvalues

• [required] Lecture Video: Eigenvectors and Eigenvalues (26:10)
• [required] MML 4.2
• [optional] 3Blue1Brown video on eigenvectors and eigenvalues

Matrix Factorization

• [required] Lecture Video: Modeling Data with Matrix Factorization (12:07)
• [required] MML 4.6-4.7
• [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

Singular Value Decomposition

• [required] Lecture Video: SVD Basics (24:56)
• [required] MML 4.5
• [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Topics:

• Random variables
• Probability density functions
• Probability mass functions
• Some useful distributions

Readings and Supplementary Material:

• [required] MML 6.0-6.2

Topics:

• Joint probability
• Independence and dependence
• Covariance
• Conditional independence

Readings and Supplementary Material:

• [required] MML 6.3-6.4

Videos:

• Basic inequalities and limit theorems
• Transforming random variables
• Univariate Gaussian distribution
• Multivariate Gaussian distribution
• Pseudo-random numbers
• Inverse transform sampling

Readings and Supplementary Material:

• [required] MML 6.7

Topics:

• Integrals as expectations
• Proving unbiasedness
• Variance of Monte Carlo estimators
• Rejection sampling
• Importance sampling
• Information theory

Readings and Supplementary Material:

• [required] MML 6.5

Videos:

• Differentiation basics
• Partial derivatives
• Best affine approximation
• Gradient and steepest ascent
• Differentiation with respect to vectors and matrices
• Useful identities

Readings and Supplementary Material:

• [required] MML 5.0-5.1
• [required] MML 5.2-5.5
• [optional] MML 5.6-5.8

Topics:

• Optimization basics
• Constrained optimization
• Lagrange multipliers
• Convex optimization
• Linear programming

Readings and Supplementary Material:

• [required] MML 7.0-7.1
• [required] MML 7.2
• [required] MML 7.3