Princeton University, Spring 2021

Contact: cos302-s21@lists.cs.princeton.eduCourse Google Calendar

Prof. Ryan Adams (OH: Mon/Wed 1:30-3pm) Zoom

TA: Joshua Aduol (OH: Tue 12:30-1:30pm, Thu 11am-12pm) Zoom

TA: Jad Rahme (OH: Tue 3-4pm, Thu 10-11am ) Zoom

TA: Geoffrey Roeder (OH: Mon 4-5pm, Fri 1:30-2:30pm) Zoom

Lab TA: Alan Chung (OH Thu 6-8pm) Zoom

Lab TA: Mwad Saleh (OH Sun 11am-1pm) Zoom

Lab TA: Kartik Shah (OH: Mon 11am-12pm, Wed 11am-12pm) Zoom

Precept 01: Jad -- Thu 9:00-9:50am Zoom

Precept 02: Joshua -- Thu 10:00-10:50am Zoom

Precept 03: Geoffrey -- Fri 12:30-1:20pm Zoom

- 3 May 2021: Solutions for HW9 and HW10 posted.
- 19 April 2021: Solutions for HW5-HW8 posted.
- 14 April 2021: HW10 is posted.
- 6 April 2021: HW9 is posted.
- 29 March 2021: HW8 is posted.
- 22 March 2021: HW7 is posted.
- 17 March 2021: HW6 is posted.
- 6 March 2021: Solutions for HW1-HW4 posted.
- 1 March 2021: HW5 is posted.
- 22 February 2021: HW4 is posted.
- 15 February 2021: HW3 is posted.
- 8 February 2021: HW2 is posted.
- 1 February 2021: HW1 is posted.
- 22 January 2021: Precept P05 (Fri 1:30-2:20pm) is cancelled.

Assignment 1 Out

- [required] Lecture Video: Introduction and Logistics (18:37)
- [required] Reading: Course Syllabus

- [required] Lecture Video: Vector Basics (15:03)
- [required] Reading: MML 2.0
- [optional] Video: 3Blue1Brown on vectors

- [required] Lecture Video: Matrix Basics (18:44)
- [required] Reading: MML 2.2
- [optional] Video: 3Blue1Brown on matrix multiplication

- [required] Lecture Video: Solving Linear Systems (36:44)
- [required] Reading: MML 2.1
- [required] Reading: MML 2.3

Week 2: February 8-12, 2021

Assignment 1 **Due 6pm ET Friday** 12 Feb 2021

Assignment 2 Out

- [required] Lecture Video: Matrix Inversion (25:06)
- [optional] Blog post: Don't invert that matrix.

- [required] Lecture Video: Vector Spaces (16:11)
- [required] MML 2.4

- [required] Lecture Video: Linear Independence, Basis, and Rank (14:00)
- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors

Week 3: February 15-19, 2021

Assignment 2 **Due 6pm ET Friday** 19 Feb 2021

Assignment 3 Out

- [required] Lecture Video: Linear maps (9:35)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on linear transformations and matrices

- [required] Lecture Video: Change of Basis (11:00)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis

- [required] Lecture Video: Norms and Inner Products (18:58)
- [required] MML 3.0-3.3

- [required] Lecture Video: Orthogonality and Projection (17:16)
- [required] MML 3.4-3.8

Week 4: February 22-26, 2021

Assignment 3 **Due 6pm ET Friday** 26 Feb 2021

Assignment 4 Out

- [required] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
- [required] MML 3.4-3.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

- [required] Lecture Video: Matrix Invariants (12:28)
- [required] MML 4.0-4.1
- [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
- [optional] 3Blue1Brown video on determinants

- [required] Lecture Video: Eigenvectors and Eigenvalues (26:10)
- [required] MML 4.2
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

Week 5: March 1-5, 2021

Assignment 4 **Due 6pm ET Friday** 5 March 2021

Assignment 5 Out

- [required] Lecture Video: Modeling Data with Matrix Factorization (12:07)
- [required] MML 4.6-4.7
- [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

- [required] Lecture Video: SVD Basics (24:56)
- [required] MML 4.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

- Catchup and review
- MIDTERM
- No precept

Assignment 5 **Due 6pm ET Friday** 19 March 2021

Assignment 6 Out

- [required] Lecture Video: Why is Probability Important in Machine Learning (6:54)
- [required] Lecture Video: Probability Spaces and Random Variables (7:02)

- [required] Lecture Video: Probability Density and Mass Functions (6:56)
- [required] Lecture Video: Some Useful Probability Distributions (5:32)
- [required] MML 6.0-6.2
- [optional] Video: 3Blue1Brown on the Binomial Distribution

Week 8: March 22-26, 2021

Assignment 7 Out

- [required] Lecture Video: Basics of Joint Probability (6:53)
- [required] MML 6.3
- [optional] Video: 3Blue1Brown on Bayes' Theorem
- [optional] Metacademy: Bayesian Machine Learning Roadmap

- [required] Lecture Video: Independence and Dependence (7:43)
- [required] MML 6.4

Week 9: March 29 - April 2, 2021

Assignment 7 **Due 6pm ET Friday** 2 April 2021

Assignment 8 Out

- [required] Lecture Video: Useful Inequalities and Limit Theorems (13:58)

- [required] Lecture Video: The Gaussian Distribution (10:13)
- [required] MML 6.5

- [required] Lecture Video: Pseudo-Random Numbers (25:48)
- [required] MML 6.7

Week 10: April 5-9, 2021

Assignment 8 **Due 6pm ET Friday** 9 April 2021

Assignment 9 Out

- [required] Lecture Video: Monte Carlo (14:59)
- [required] MML 6.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

- [required] Lecture Video: Information Theory Basics (16:22)

Assignment 9 **Due 6pm ET Friday** 16 April 2021
Assignment 10 Out

- [required] Lecture Video: Why is Differentiation Important to Machine Learning? (3:09)
- [required] Lecture Video: Differentiation Basics (4:44)
- [required] Lecture Video: Partial Derivatives (4:17)
- [required] Lecture Video: Best Affine Approximation (5:26)
- [required] Lecture Video: Practical Differentiation (31:11)
- [required] MML 5.0-5.5
- [optional] MML 5.6-5.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

Week 12: April 19-23 2021

Assignment 10 **Due 6pm ET Friday** 23 April 2021

- [required] Lecture Video: Why is the Gradient the Direction of Steepest Ascent? (2:41)
- [required] Lecture Video: Optimization Basics (8:05)
- [required] Lecture Video: Convex Optimization (21:32)
- [required] MML 7.0-7.3

- Assignment 1 -- Out Mon 1 Feb, Due Fri 12 Feb at 6:00pm [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission, hw1-solutions.pdf]
- Assignment 2 -- Out Mon 8 Feb, Due Fri 19 Feb at 6:00pm [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission, hw2-solutions.pdf]
- Assignment 3 -- Out Mon 15 Feb, Due Fri 26 Feb at 6:00pm [hw3.pdf, Overleaf template, hw3.tex, cos302.cls, coords.pkl, mnist2000.pkl, Gradescope submission, hw3-solutions.pdf]
- Assignment 4 -- Out Mon 22 Feb, Due Fri 5 Mar at 6:00pm [hw4.pdf, Overleaf template, hw4.tex, cos302.cls, mnist2000.pkl, Gradescope submission, hw4-solutions.pdf]
- Assignment 5 -- Out Mon 1 Mar, Due Fri 19 Mar at 6:00pm [hw5.pdf, Overleaf template, hw5.tex, cos302.cls, nyt.pkl.gz, dog_names1000.txt, Gradescope submission, hw5-solutions.pdf]
- Assignment 6 -- Out Wed 17 Mar, Due Fri 26 Mar at 6:00pm [hw6.pdf, Overleaf template, hw6.tex, cos302.cls, Gradescope submission, hw6-solutions.pdf]
- Assignment 7 -- Out Mon 22 Mar, Due Fri 2 Apr at 6:00pm [hw7.pdf, Overleaf template, hw7.tex, cos302.cls, Gradescope submission, hw7-solutions.pdf]
- Assignment 8 -- Out Mon 29 Mar, Due Fri 9 Apr at 6:00pm [hw8.pdf, Overleaf template, hw8.tex, cos302.cls, Gradescope submission, hw8-solutions.pdf]
- Assignment 9 -- Out Tue 6 Apr, Due Fri 16 Apr at 6:00pm [hw9.pdf, Overleaf template, hw9.tex, cos302.cls, mnist_full.pkl.gz, Gradescope submission, hw9-solutions.pdf]
- Assignment 10 -- Out Wed 14 Apr, Due Tue 27 Apr at 6:00pm [hw10.pdf, Overleaf template, hw10.tex, cos302.cls, Gradescope submission, hw10-solutions.pdf]

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**Are precepts required?**No, but you should attend precept.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.