Princeton University, Fall 2022

Lecture: Mon/Weds 12:30-1:20pm in Friend Center 008Contact: cos302-f22@lists.cs.princeton.edu

Prof. Ryan Adams (OH: Mon/Wed 1:30-3pm in COS 411)

TA: Alex Guerra (OH: Tue 4-6pm, COS 401)

TA: Nick Richardson (OH: Mon 3-5pm, COS 003)

Lab TA: Indu Panigrahi (OH: Sat 10am-12pm, Friend 010)

Precept 01 (Nick): Thu 10:00-10:50am in Friend Center 110

Precept 02 (Alex): Fri 12:30-1:20pm in Friend Center 110

- 10 Dec 2022: Final exam practice questions here.
- 7 Dec 2022: HW11 is out, due Friday 16 December (Dean's date).
- 1 Dec 2022: HW10 is out, due Friday 9 December.
- 23 Nov 2022: HW9 is out, due Friday 2 December.
- 7 Nov 2022: HW8 is out, due Wednesday 16 November.
- 30 Oct 2022: HW7 is out, due Wednesday 9 November.
- 15 Oct 2022: HW6 is out, due Wednesday 2 November.
- 1 Oct 2022: HW5 is out, due Friday 14 October.
- 21 Sept 2022: HW4 is out, due Friday 7 October.
- 14 Sept 2022: HW3 is out, due 28 September.
- 7 Sept 2022: HW1 and HW2 are out, due 14 Sept and 21 Sept, respectively.

Assignment 1 Out

Assignment 2 Out

- [required] Reading: Course Syllabus

- [required] Reading: MML 2.0
- [optional] Lecture Video: Vector Basics (15:03)
- [optional] Video: 3Blue1Brown on vectors

- [required] Reading: MML 2.2
- [optional] Lecture Video: Matrix Basics (18:44)
- [optional] Video: 3Blue1Brown on matrix multiplication

Week 2: 12-16 September

Assignment 1 **Due 6pm ET Wednesday** 14 Sept 2022

Assignment 3 Out

- [required] Reading: MML 2.1
- [required] Reading: MML 2.3
- [optional] Lecture Video: Solving Linear Systems (36:44)

- [optional] Blog post: Don't invert that matrix.
- [optional] Lecture Video: Matrix Inversion (25:06)

- [required] MML 2.4
- [optional] Lecture Video: Vector Spaces (16:11)

Week 3: 19-23 September

Assignment 2 **Due 6pm ET Wednesday** 21 Sept 2021

Assignment 4 Out

- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors
- [optional] Lecture Video: Linear Independence, Basis, and Rank (14:00)

- [optional] Lecture Video: Linear maps (9:35)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on linear transformations and matrices

- [optional] Lecture Video: Change of Basis (11:00)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis

- [optional] Lecture Video: Norms and Inner Products (18:58)
- [required] MML 3.0-3.3

Week 4: 26-30 September

Assignment 3 **Due 6pm ET Wednesday** 28 Sept 2022

Assignment 5 Out

- [optional] Lecture Video: Orthogonality and Projection (17:16)
- [required] MML 3.4-3.8

- [optional] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
- [required] MML 3.4-3.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

- [optional] Lecture Video: Matrix Invariants (12:28)
- [required] MML 4.0-4.1
- [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
- [optional] 3Blue1Brown video on determinants

Week 5: 3-7 October

Assignment 4 **Due 6pm ET Friday** 7 Oct 2022

- [optional] Lecture Video: Eigenvectors and Eigenvalues (26:10)
- [required] MML 4.2
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

- [optional] Lecture Video: Modeling Data with Matrix Factorization (12:07)
- [required] MML 4.6-4.7
- [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

**Midterm Exam in class Oct 12**

No precept Thurs/Fri.

Assignment 5 **Due 6pm ET Friday** 14 Oct 2022

Assignment 6 Out

- [optional] Lecture Video: SVD Basics (24:56)
- [required] MML 4.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Assignment 7 Out

- [optional] Lecture Video: Why is Probability Important in Machine Learning (6:54)
- [optional] Lecture Video: Probability Spaces and Random Variables (7:02)

- [optional] Lecture Video: Probability Density and Mass Functions (6:56)
- [optional] Lecture Video: Some Useful Probability Distributions (5:32)
- [required] MML 6.0-6.2
- [optional] Video: 3Blue1Brown on the Binomial Distribution

Week 8: 31 October to 4 November

Assignment 6 **Due 6pm ET Wednesday** 2 Nov 2022

Assignment 8 Out

- [optional] Lecture Video: Basics of Joint Probability (6:53)
- [required] MML 6.3
- [optional] Video: 3Blue1Brown on Bayes' Theorem
- [optional] Metacademy: Bayesian Machine Learning Roadmap

- [optional] Lecture Video: Independence and Dependence (7:43)
- [required] MML 6.4

- [optional] Lecture Video: The Gaussian Distribution (10:13)
- [required] MML 6.5

Week 9: 7-11 November

Assignment 7 **Due 6pm ET Wednesday** 9 Nov 2022

Assignment 9/a> Out

- [required] MML 6.7

- [required] MML 6.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

- [optional] Lecture Video: Information Theory Basics (16:22)

Assignment 8 **Due 6pm ET Wednesday** 16 Nov 2022

Assignment 10 Out

- [optional] Lecture Video: Why is Differentiation Important to Machine Learning? (3:09)
- [optional] Lecture Video: Differentiation Basics (4:44)
- [optional] Lecture Video: Partial Derivatives (4:17)
- [optional] Lecture Video: Best Affine Approximation (5:26)
- [required] MML 5.0-5.5
- [optional] MML 5.6-5.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

Week 11: 21-22 November (short week)

No lecture Weds, no precept Thurs/Fri.

- [optional] Lecture Video: Why is the Gradient the Direction of Steepest Ascent? (2:41)
- [optional] Lecture Video: Optimization Basics (8:05)
- [required] MML 7.0

Week 12: 28 November to 2 December

- [required] MML 7.1

- [required] MML 5.6

- [required] MML 7.2

Week 13: 5 December to 9 December

- [optional] Video: Mathematical Monk on Newton's method

- [required] MML 7.3
- [optional] Lecture Video: Convex Optimization (21:32)

Reading Week: 12 December to 16 December

Assignment 11 **Due 6pm ET Wednesday** 14 Dec 2022

- Assignment 1 -- Out Wed 7 Sept, Due Wed 14 Sept at 6:00pm [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission, hw1-solutions.pdf ]
- Assignment 2 -- Out Wed 7 Sept, Due Wed 21 Sept at 6:00pm [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission, hw2-solutions.pdf ]
- Assignment 3 -- Out Wed 14 Sept, Due Wed 28 Sept at 6:00pm [hw3.pdf, Overleaf template, hw3.tex, mnist2000.pkl, coords.pkl, cos302.cls, Gradescope submission, hw3-solutions.pdf ]
- Assignment 4 -- Out Wed 21 Sept, Due Fri 7 October at 6:00pm [hw4.pdf, Overleaf template, hw4.tex, mnist2000.pkl, cos302.cls, Gradescope submission, hw4-solutions.pdf ]
- Assignment 5 -- Out Sat 1 Oct, Due Fri 14 October at 6:00pm [hw5.pdf, Overleaf template, hw5.tex, dog_names1000.txt, nyt_data.txt.gz, cos302.cls, Gradescope submission, hw5-solutions.pdf ]
- Assignment 6 -- Out Sat 15 Oct, Due Wed 2 November at 6:00pm [hw6.pdf, Overleaf template, hw6.tex, cos302.cls, Gradescope submission, hw6-solutions.pdf ]
- Assignment 7 -- Out Sun 30 Oct, Due Wed 9 November at 6:00pm [hw7.pdf, Overleaf template, hw7.tex, cos302.cls, Gradescope submission, hw7-solutions.pdf ]
- Assignment 8 -- Out Mon 7 Nov, Due Wed 16 November at 6:00pm [hw8.pdf, Overleaf template, hw8.tex, cos302.cls, Gradescope submission, hw8-solutions.pdf ]
- Assignment 9 -- Out Wed 23 Nov, Due Fri 2 December at 6:00pm [hw9.pdf, Overleaf template, hw9.tex, cos302.cls, Gradescope submission, hw9-solutions.pdf ]
- Assignment 10 -- Out Thu 1 Dec, Due Fri 9 December at 6:00pm [hw10.pdf, Overleaf template, hw10.tex, cos302.cls, Gradescope submission, hw10-solutions.pdf ]
- Assignment 11 -- Out Wed 7 Dec, Due Fri 16 December at 6:00pm (Dean's date, so no extensions possible) [hw11.pdf, Overleaf template, hw11.tex, cos302.cls, Gradescope submission, hw11-solutions.pdf ]

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Can I just watch the old videos rather than come to lecture?**You are responsible for the material I present in lecture, and you should come to lecture. The videos are there to be helpful for review.**Are precepts required?**No, but you should attend precept. Novel material will be presented in them.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.