Princeton University, Spring 2021

Contact: cos302-s21@lists.cs.princeton.eduCourse Google Calendar

Prof. Ryan Adams (OH: Mon/Wed 1:30-3pm) Zoom

TA: Joshua Aduol (OH: Tue 12:30-1:30pm, Thu 11am-12pm) Zoom

TA: Jad Rahme (OH: Tue 3-4pm, Thu 10-11am ) Zoom

TA: Geoffrey Roeder (OH: Mon 4-5pm, Fri 1:30-2:30pm) Zoom

Lab TA: Alan Chung (OH Thu 6-8pm) Zoom

Lab TA: Mwad Saleh (OH Sun 11am-1pm) Zoom

Lab TA: Kartik Shah (OH: Mon 11am-12pm, Wed 11am-12pm) Zoom

Precept 01: Jad -- Thu 9:00-9:50am Zoom

Precept 02: Joshua -- Thu 10:00-10:50am Zoom

Precept 03: Geoffrey -- Fri 12:30-1:20pm Zoom

- 1 March 2021: HW5 is posted.
- 22 February 2021: HW4 is posted.
- 15 February 2021: HW3 is posted.
- 8 February 2021: HW2 is posted.
- 1 February 2021: HW1 is posted.
- 22 January 2021: Precept P05 (Fri 1:30-2:20pm) is cancelled.

Assignment 1 Out

- [required] Lecture Video: Introduction and Logistics (18:37)
- [required] Reading: Course Syllabus

- [required] Lecture Video: Vector Basics (15:03)
- [required] Reading: MML 2.0
- [optional] Video: 3Blue1Brown on vectors

- [required] Lecture Video: Matrix Basics (18:44)
- [required] Reading: MML 2.2
- [optional] Video: 3Blue1Brown on matrix multiplication

- [required] Lecture Video: Solving Linear Systems (36:44)
- [required] Reading: MML 2.1
- [required] Reading: MML 2.3

Week 2: February 8-12, 2021

Assignment 1 **Due 6pm ET Friday** 12 Feb 2021

Assignment 2 Out

- [required] Lecture Video: Matrix Inversion (25:06)
- [optional] Blog post: Don't invert that matrix.

- [required] Lecture Video: Vector Spaces (16:11)
- [required] MML 2.4

- [required] Lecture Video: Linear Independence, Basis, and Rank (14:00)
- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors

Week 3: February 15-19, 2021

Assignment 2 **Due 6pm ET Friday** 19 Feb 2021

Assignment 3 Out

- [required] Lecture Video: Linear maps (9:35)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on linear transformations and matrices

- [required] Lecture Video: Change of Basis (11:00)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis

- [required] Lecture Video: Norms and Inner Products (18:58)
- [required] MML 3.0-3.3

- [required] Lecture Video: Orthogonality and Projection (17:16)
- [required] MML 3.4-3.8

Week 4: February 22-26, 2021

Assignment 3 **Due 6pm ET Friday** 26 Feb 2021

Assignment 4 Out

- [required] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
- [required] MML 3.4-3.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

- [required] Lecture Video: Matrix Invariants (12:28)
- [required] MML 4.0-4.1
- [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
- [optional] 3Blue1Brown video on determinants

- [required] Lecture Video: Eigenvectors and Eigenvalues (26:10)
- [required] MML 4.2
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

Week 5: March 1-5, 2021

Assignment 4 **Due 6pm ET Friday** 5 March 2021

Assignment 5 Out

- [required] Lecture Video: Modeling Data with Matrix Factorization (12:07)
- [required] MML 4.6-4.7
- [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

- [required] Lecture Video: SVD Basics (24:56)
- [required] MML 4.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

- Catchup and review
- MIDTERM
- No precept

Assignment 5 **Due 6pm ET Friday** 19 March 2021

Assignment 6 Out

- Random variables
- Probability density functions
- Probability mass functions
- Some useful distributions

- [required] MML 6.0-6.2

Week 8: March 22-26, 2021

Assignment 7 Out

- Joint probability
- Independence and dependence
- Covariance
- Conditional independence

- [required] MML 6.3-6.4

Week 9: March 29 - April 2, 2021

Assignment 7 **Due 6pm ET Friday** 2 April 2021

Assignment 8 Out

- Basic inequalities and limit theorems
- Transforming random variables
- Univariate Gaussian distribution
- Multivariate Gaussian distribution
- Pseudo-random numbers
- Inverse transform sampling

- [required] MML 6.7

Week 10: April 5-9, 2021

Assignment 8 **Due 6pm ET Friday** 9 April 2021

Assignment 9 Out

- Integrals as expectations
- Proving unbiasedness
- Variance of Monte Carlo estimators
- Rejection sampling
- Importance sampling
- Information theory

- [required] MML 6.5

Assignment 9 **Due 6pm ET Friday** 16 April 2021
Assignment 10 Out

- Differentiation basics
- Partial derivatives
- Best affine approximation
- Gradient and steepest ascent
- Differentiation with respect to vectors and matrices
- Useful identities

- [required] MML 5.0-5.1
- [required] MML 5.2-5.5
- [optional] MML 5.6-5.8

Assignment 10 **Due 6pm ET Friday** 23 April 2021

- Optimization basics
- Constrained optimization
- Lagrange multipliers
- Convex optimization
- Linear programming

- [required] MML 7.0-7.1
- [required] MML 7.2
- [required] MML 7.3

- Assignment 1 -- Out Mon 1 Feb, Due Fri 12 Feb at 6:00pm [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission]
- Assignment 2 -- Out Mon 8 Feb, Due Fri 19 Feb at 6:00pm [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission]
- Assignment 3 -- Out Mon 15 Feb, Due Fri 26 Feb at 6:00pm [hw3.pdf, Overleaf template, hw3.tex, cos302.cls, coords.pkl, mnist2000.pkl, Gradescope submission]
- Assignment 4 -- Out Mon 22 Feb, Due Fri 5 Mar at 6:00pm [hw4.pdf, Overleaf template, hw4.tex, cos302.cls, mnist2000.pkl, Gradescope submission]
- Assignment 5 -- Out Mon 1 Mar, Due Fri 19 Mar at 6:00pm [hw5.pdf, Overleaf template, hw5.tex, cos302.cls, nyt.pkl.gz, dog_names1000.txt, Gradescope submission]

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**Are precepts required?**No, but you should attend precept.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.