Princeton University, Fall 2022

Lecture: Mon/Weds 12:30-1:20pm in Friend Center 008Contact: cos302-f22@lists.cs.princeton.edu

Prof. Ryan Adams (OH: Mon/Wed 1:30-3pm in COS 411)

TA: Alex Guerra (OH: Tue 4-6pm, COS 401)

TA: Nick Richardson (OH: Mon 3-5pm, COS 003)

Lab TA: Indu Panigrahi (OH: Sat 10am-12pm, Friend 010)

Precept 01 (Nick): Thu 10:00-10:50am in Friend Center 110

Precept 02 (Alex): Fri 12:30-1:20pm in Friend Center 110

- 1 Oct 2022: HW5 is out, due Friday 14 October.
- 21 Sept 2022: HW4 is out, due Friday 7 October.
- 14 Sept 2022: HW3 is out, due 28 September.
- 7 Sept 2022: HW1 and HW2 are out, due 14 Sept and 21 Sept, respectively.

Assignment 1 Out

Assignment 2 Out

- [required] Reading: Course Syllabus

- [required] Reading: MML 2.0
- [optional] Lecture Video: Vector Basics (15:03)
- [optional] Video: 3Blue1Brown on vectors

- [required] Reading: MML 2.2
- [optional] Lecture Video: Matrix Basics (18:44)
- [optional] Video: 3Blue1Brown on matrix multiplication

Week 2: 12-16 September

Assignment 1 **Due 6pm ET Wednesday** 14 Sept 2022

Assignment 3 Out

- [required] Reading: MML 2.1
- [required] Reading: MML 2.3
- [optional] Lecture Video: Solving Linear Systems (36:44)

- [optional] Blog post: Don't invert that matrix.
- [optional] Lecture Video: Matrix Inversion (25:06)

- [required] MML 2.4
- [optional] Lecture Video: Vector Spaces (16:11)

Week 3: 19-23 September

Assignment 2 **Due 6pm ET Wednesday** 21 Sept 2021

Assignment 4 Out

- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors
- [optional] Lecture Video: Linear Independence, Basis, and Rank (14:00)

- [optional] Lecture Video: Linear maps (9:35)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on linear transformations and matrices

- [optional] Lecture Video: Change of Basis (11:00)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis

- [optional] Lecture Video: Norms and Inner Products (18:58)
- [required] MML 3.0-3.3

Week 4: 26-30 September

Assignment 3 **Due 6pm ET Wednesday** 28 Sept 2022

Assignment 5 Out

- [optional] Lecture Video: Orthogonality and Projection (17:16)
- [required] MML 3.4-3.8

- [optional] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
- [required] MML 3.4-3.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

- [optional] Lecture Video: Matrix Invariants (12:28)
- [required] MML 4.0-4.1
- [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
- [optional] 3Blue1Brown video on determinants

Week 5: 3-7 October

Assignment 4 **Due 6pm ET Friday** 7 Oct 2022

Assignment 6 Out

- [optional] Lecture Video: Eigenvectors and Eigenvalues (26:10)
- [required] MML 4.2
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

- [optional] Lecture Video: Modeling Data with Matrix Factorization (12:07)
- [required] MML 4.6-4.7
- [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

**Midterm Exam in class Oct 12**

No precept Thurs/Fri.

Assignment 5 **Due 6pm ET Friday** 14 Oct 2022

Assignment 7 Out

- [optional] Lecture Video: SVD Basics (24:56)
- [required] MML 4.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Assignment 6 **Due 6pm ET Wednesday** 26 Oct 2022

Assignment 8 Out

- [optional] Lecture Video: Why is Probability Important in Machine Learning (6:54)
- [optional] Lecture Video: Probability Spaces and Random Variables (7:02)

- [optional] Lecture Video: Probability Density and Mass Functions (6:56)
- [optional] Lecture Video: Some Useful Probability Distributions (5:32)
- [required] MML 6.0-6.2
- [optional] Video: 3Blue1Brown on the Binomial Distribution

Week 8: 31 October to 4 November

Assignment 7 **Due 6pm ET Wednesday** 2 Nov 2022

Assignment 9 Out

- [optional] Lecture Video: Basics of Joint Probability (6:53)
- [required] MML 6.3
- [optional] Video: 3Blue1Brown on Bayes' Theorem
- [optional] Metacademy: Bayesian Machine Learning Roadmap

- [optional] Lecture Video: Independence and Dependence (7:43)
- [required] MML 6.4

- [optional] Lecture Video: The Gaussian Distribution (10:13)
- [required] MML 6.5

Week 9: 7-11 November

Assignment 8 **Due 6pm ET Wednesday** 9 Nov 2022

Assignment 10/a> Out

- [required] MML 6.7

- [required] MML 6.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

- [optional] Lecture Video: Information Theory Basics (16:22)

Week 10: 14-18 November

Assignment 9 **Due 6pm ET Wednesday** 16 Nov 2022

Assignment 11 Out

- [optional] Lecture Video: Why is Differentiation Important to Machine Learning? (3:09)
- [optional] Lecture Video: Differentiation Basics (4:44)
- [optional] Lecture Video: Partial Derivatives (4:17)
- [optional] Lecture Video: Best Affine Approximation (5:26)
- [required] MML 5.0-5.5
- [optional] MML 5.6-5.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

No lecture Weds, no precept Thurs/Fri.

- [optional] Lecture Video: Why is the Gradient the Direction of Steepest Ascent? (2:41)
- [optional] Lecture Video: Optimization Basics (8:05)
- [required] MML 7.0

Week 12: 28 November to 2 December

Assignment 10 **Due 6pm ET Wednesday** 30 Nov 2022

- [required] MML 7.1

- [required] MML 5.6

- [required] MML 7.2

Week 13: 5 December to 7 December

Assignment 11 **Due 6pm ET Wednesday** 7 Dec 2022

- [optional] Video: Mathematical Monk on Newton's method

- [required] MML 7.3
- [optional] Lecture Video: Convex Optimization (21:32)

- Assignment 1 -- Out Wed 7 Sept, Due Wed 14 Sept at 6:00pm [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission, hw1-solutions.pdf ]
- Assignment 2 -- Out Wed 7 Sept, Due Wed 21 Sept at 6:00pm [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission, hw2-solutions.pdf ]
- Assignment 3 -- Out Wed 14 Sept, Due Wed 28 Sept at 6:00pm [hw3.pdf, Overleaf template, hw3.tex, mnist2000.pkl, coords.pkl, cos302.cls, Gradescope submission ]
- Assignment 4 -- Out Wed 21 Sept, Due Fri 7 October at 6:00pm [hw4.pdf, Overleaf template, hw4.tex, mnist2000.pkl, cos302.cls, Gradescope submission ]
- Assignment 5 -- Out Sat 1 Oct, Due Fri 14 October at 6:00pm [hw5.pdf, Overleaf template, hw5.tex, dog_names1000.txt, nyt_data.txt.gz, cos302.cls, Gradescope submission ]

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Can I just watch the old videos rather than come to lecture?**You are responsible for the material I present in lecture, and you should come to lecture. The videos are there to be helpful for review.**Are precepts required?**No, but you should attend precept. Novel material will be presented in them.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.