COS 302 / SML 305: Mathematics for Numerical Computing and Machine Learning

Vectors

• [required] Reading: MML 2.0
• [optional] Sp21 Lecture by Ryan Adams: Vector Basics (15:03)
• [optional] Video: 3Blue1Brown on vectors

Matrices

• [required] Reading: MML 2.2
• [optional] Sp21 Lecture by Ryan Adams: Matrix Basics (18:44)
• [optional] Video: 3Blue1Brown on matrix multiplication

Systems of Linear Equations

• [required] Reading: MML 2.1
• [required] Reading: MML 2.3
• [optional] Sp21 Lecture by Ryan Adams: Solving Linear Systems (36:44)

Matrix Inversion

• [optional] Sp21 Lecture by Ryan Adams: Matrix Inversion (25:06)
• [optional] Blog post: Don't invert that matrix.

Vector Spaces

• [required] MML 2.4
• [optional] Sp21 Lecture by Ryan Adams: Vector Spaces (16:11)

Linear Independence, Basis and Rank

• [required] MML 2.5-2.6
• [optional] Sp21 Lecture by Ryan Adams: Linear Independence, Basis, and Rank (14:00)
• [optional] 3Blue1Brown video on basis vectors

Linear Maps

• [required] MML 2.7-2.8
• [required] Sp21 Lecture by Ryan Adams: Linear maps (9:35)
• [optional] 3Blue1Brown video on linear transformations and matrices

Change of Basis

• [required] MML 2.7-2.8
• [optional] Sp21 Lecture by Ryan Adams: Change of Basis (11:00)
• [optional] 3Blue1Brown video on change of basis

Norms and Inner Products

• [required] MML 3.0-3.3
• [optional] Sp21 Lecture by Ryan Adams: Norms and Inner Products (18:58)

Orthogonality

• [required] MML 3.4-3.8
• [optional] Sp21 Lecture by Ryan Adams: Orthogonality and Projection (17:16)

Gram-Schmidt Orthogonalization

• [required] MML 3.4-3.8
• [optional] Sp21 Lecture by Ryan Adams: Gram-Schmidt Orthogonalization (15:15)
• [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

Determinant and Trace

• [required] MML 4.0-4.1
• [optional] Sp21 Lecture by Ryan Adams: Matrix Invariants (12:28)
• [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
• [optional] 3Blue1Brown video on determinants

Eigenvectors and Eigenvalues

• [required] MML 4.2
• [optional] Sp21 Lecture by Ryan Adams: Eigenvectors and Eigenvalues (26:10)
• [optional] 3Blue1Brown video on eigenvectors and eigenvalues

Matrix Factorization

• [optional] Sp21 Lecture by Ryan Adams: Modeling Data with Matrix Factorization (12:07)
• [required] MML 4.6-4.7
• [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

Singular Value Decomposition

• [optional] Sp21 Lecture by Ryan Adams: SVD Basics (24:56)
• [required] MML 4.5
• [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Random Variables

• [optional] Sp21 Lecture by Ryan Adams: Why is Probability Important in Machine Learning (6:54)
• [optional] Sp21 Lecture by Ryan Adams: Probability Spaces and Random Variables (7:02)
• [required] MML 6.0

Probability Density and Mass Functions

• [optional] Sp21 Lecture by Ryan Adams: Probability Density and Mass Functions (6:56)
• [optional] Sp21 Lecture by Ryan Adams: Some Useful Probability Distributions (5:32)
• [required] MML 6.1-6.2
• [optional] Video: 3Blue1Brown on the Binomial Distribution

Joint Probability

• [optional] Sp21 Lecture by Ryan Adams: Basics of Joint Probability (6:53)
• [required] MML 6.3
• [optional] Video: 3Blue1Brown on Bayes' Theorem
• [optional] Metacademy: Bayesian Machine Learning Roadmap

Independence and Dependence

• [optional] Sp21 Lecture by Ryan Adams: Independence and Dependence (7:43)
• [required] MML 6.4

Inequalities and Limits

• [optional] Sp21 Lecture by Ryan Adams: Useful Inequalities and Limit Theorems (13:58)

The Gaussian Distribution

• [optional] Sp21 Lecture by Ryan Adams: The Gaussian Distribution (10:13)
• [required] MML 6.5

The Gaussian Distribution: Continued

Pseudo-Random Numbers

• [required] MML 6.7

Monte Carlo

• [required] MML 6.5
• [optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

Information Theory

• [optional] Sp21 Lecture by Ryan Adams: Information Theory Basics (16:22)

Differentiation

• [required] MML 5.0-5.5
• [optional] Sp21 Lecture by Ryan Adams: Why is Differentiation Important to Machine Learning? (3:09)
• [optional] Sp21 Lecture by Ryan Adams: Differentiation Basics (4:44)
• [optional] Sp21 Lecture by Ryan Adams: Partial Derivatives (4:17)
• [optional] Sp21 Lecture by Ryan Adams: Best Affine Approximation (5:26)

Differentiation: Continued

• [optional] Sp21 Lecture by Ryan Adams: Practical Differentiation (31:11)
• [optional] MML 5.6-5.8
• [optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

Optimization

• [required] MML 7.0-7.2
• [optional] Sp21 Lecture by Ryan Adams: Why is the Gradient the Direction of Steepest Ascent? (2:41)
• [optional] Sp21 Lecture by Ryan Adams: Optimization Basics (8:05)

Convex Optimization

• [required] MML 7.3
• [optional] Sp21 Lecture by Ryan Adams: Convex Optimization (21:32)