Princeton University, Fall 2023

Lecture: Mon/Weds 11:00am-12:20pm in Friend 006Prof. Ryan Adams (OH: Weds 1:30-3pm in COS 411)

TA: David Xu(OH: Tues 2-4pm, COS 003)

TA: Andre Niyongabo Rubungo (OH: Mon 1:30-2:30pm, common space outside of CS 241; Mon 5-6pm, common space outside of CS 241)

TA: Haichen Dong (OH: Mon 7-8pm, zoom; Tue 11am-12pm, Friend 010)

Lab TA: Aditya Palaparthi (OH: Sun 10-11am, zoom; Mon 3-4pm, common space outside of CS 241; Tues 8-9pm zoom)

Lab TA: Kevin Kim (OH Mon 8-9pm, Lewis Library 406)

Lab TA: Evan Wang (OH: Tues 4:30-6:30pm, zoom)

Precept 01 (Haichen): Thu 10:00-10:50am in Friend 110

Precept 02 (Andre): Thu 12:30-1:20pm in Sherrerd H 001

Precept 03 (David): Thu 9:00-9:50am in Friend 110

- 30 November 2023: Homework 11 is posted, due Friday 15 Dec (Dean's Date).
- 15 November 2023: Homework 10 is posted, due Weds 6 Dec.
- 7 November 2023: Homework 9 is posted, due Weds 29 Nov.
- 2 November 2023: Homework 8 is posted, due Weds 15 Nov.
- 26 October 2023: Homework 7 is posted, due Weds 8 Nov.

Assignment 1 Out

Assignment 2 Out

- [required] Reading: Course Syllabus

- [required] Reading: MML 2.0
- [optional] Lecture Video: Vector Basics (15:03)
- [optional] Video: 3Blue1Brown on vectors

- [required] Reading: MML 2.2
- [optional] Lecture Video: Matrix Basics (18:44)
- [optional] Video: 3Blue1Brown on matrix multiplication

Week 2: 11-15 September

Assignment 1 **Due 6pm ET Wednesday** 13 Sept 2023

Assignment 3 Out

- [required] Reading: MML 2.1
- [required] Reading: MML 2.3
- [optional] Lecture Video: Solving Linear Systems (36:44)

- [optional] Blog post: Don't invert that matrix.
- [optional] Lecture Video: Matrix Inversion (25:06)

- [required] MML 2.4
- [optional] Lecture Video: Vector Spaces (16:11)

Week 3: 18-22 September

Assignment 2 **Due 6pm ET Wednesday** 20 Sept 2023

Assignment 4 Out

- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors
- [optional] Lecture Video: Linear Independence, Basis, and Rank (14:00)

- [optional] Lecture Video: Linear maps (9:35)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on linear transformations and matrices

- [optional] Lecture Video: Change of Basis (11:00)
- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis

- [optional] Lecture Video: Norms and Inner Products (18:58)
- [required] MML 3.0-3.3

Week 4: 25-29 September

Assignment 3 **Due 6pm ET Wednesday** 27 Sept 2023

Assignment 5 Out

- [optional] Lecture Video: Orthogonality and Projection (17:16)
- [required] MML 3.4-3.8

- [optional] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
- [required] MML 3.4-3.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

- [optional] Lecture Video: Matrix Invariants (12:28)
- [required] MML 4.0-4.1
- [optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
- [optional] 3Blue1Brown video on determinants

Week 5: 2-6 October

Assignment 4 **Due 6pm ET Wednesday** 4 Oct 2023

Assignment 6 Out

- [optional] Lecture Video: Eigenvectors and Eigenvalues (26:10)
- [required] MML 4.2
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

- [optional] Lecture Video: Modeling Data with Matrix Factorization (12:07)
- [required] MML 4.6-4.7
- [optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

**Midterm Exam in class Oct 11**

No precept Thurs/Fri.

Assignment 7 Out

- [optional] Lecture Video: SVD Basics (24:56)
- [required] MML 4.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Assignment 5 **Due 6pm ET Wednesday**25 Oct 2023

- [optional] Lecture Video: Why is Probability Important in Machine Learning (6:54)
- [optional] Lecture Video: Probability Spaces and Random Variables (7:02)

- [optional] Lecture Video: Probability Density and Mass Functions (6:56)
- [optional] Lecture Video: Some Useful Probability Distributions (5:32)
- [required] MML 6.0-6.2
- [optional] Video: 3Blue1Brown on the Binomial Distribution

Week 8: 30 October to 3 November

Assignment 6 **Due 6pm ET Wednesday** 1 Nov 2023

Assignment 8 Out

- [optional] Lecture Video: Basics of Joint Probability (6:53)
- [required] MML 6.3
- [optional] Video: 3Blue1Brown on Bayes' Theorem
- [optional] Metacademy: Bayesian Machine Learning Roadmap

- [optional] Lecture Video: Independence and Dependence (7:43)
- [required] MML 6.4

- [optional] Lecture Video: The Gaussian Distribution (10:13)
- [required] MML 6.5

Week 9: 6-10 November

Assignment 7 **Due 6pm ET Wednesday**8 Nov 2023

Assignment 9/a> Out

- [required] MML 6.7

- [required] MML 6.5
- [optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

- [optional] Lecture Video: Information Theory Basics (16:22)

Assignment 8 **Due 6pm ET Wednesday** 15 Nov 2023

Assignment 10 Out

- [optional] Lecture Video: Why is Differentiation Important to Machine Learning? (3:09)
- [optional] Lecture Video: Differentiation Basics (4:44)
- [optional] Lecture Video: Partial Derivatives (4:17)
- [optional] Lecture Video: Best Affine Approximation (5:26)
- [required] MML 5.0-5.5
- [optional] MML 5.6-5.8
- [optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

Week 11: 20-22 November (short week)

No lecture Weds, no precept Thurs/Fri.

- [optional] Lecture Video: Why is the Gradient the Direction of Steepest Ascent? (2:41)
- [optional] Lecture Video: Optimization Basics (8:05)
- [required] MML 7.0

Week 12: 27 November to 1 December

Assignment 9 **Due 6pm ET Wednesday** 29 Nov 2023

Assignment 11 Out

- [required] MML 7.1

- [required] MML 5.6

- [required] MML 7.2

Week 13: 4 December to 8 December

Assignment 10 **Due 6pm ET Wednesday** 6 Dec 2023

- [optional] Video: Mathematical Monk on Newton's method

- [required] MML 7.3
- [optional] Lecture Video: Convex Optimization (21:32)

Reading Week: 11 December to 15 December

Assignment 11 **Due 6pm ET Friday** 15 Dec 2023 (Dean's Date)

- Assignment 1 -- Out Wed 7 Sept, Due Wed 13 Sept at 6:00pm [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission, hw1-solutions.pdf ]
- Assignment 2 -- Out Wed 6 Sept, Due Wed 20 Sept at 6:00pm [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission, hw2-solutions.pdf ]
- Assignment 3 -- Out Wed 13 Sept, Due Wed 27 Sept at 6:00pm [hw3.pdf, Overleaf template, hw3.tex, mnist2000.pkl, coords.pkl, cos302.cls, Gradescope submission, hw3-solutions.pdf ]
- Assignment 4 -- Out Wed 20 Sept, Due Wed 4 October at 6:00pm [hw4.pdf, Overleaf template, hw4.tex, mnist2000.pkl, cos302.cls, Gradescope submission ]
- Assignment 5 -- Out Mon 2 Oct, Due
~~Wed 25 October at 6:00pm~~Fri 27 October at 6:00pm [hw5.pdf, Overleaf template, hw5.tex, nyt.pkl.gz, cos302.cls, Gradescope submission ] - Assignment 6 -- Out Mon 23 Oct, Due Wed 1 November at 6:00pm [hw6.pdf, Overleaf template, hw6.tex, cos302.cls, Gradescope submission ]
- Assignment 7 -- Out Thu 26 Oct, Due Wed 8 November at 6:00pm [hw7.pdf, Overleaf template, hw7.tex, cos302.cls, Gradescope submission ]
- Assignment 8 -- Out Thu 2 Nov, Due Wed 15 November at 6:00pm [hw8.pdf, Overleaf template, hw8.tex, cos302.cls, Gradescope submission ]
- Assignment 9 -- Out Tue 7 Nov, Due Wed 29 November at 6:00pm [hw9.pdf, Overleaf template, hw9.tex, cos302.cls, Gradescope submission ]
- Assignment 10 -- Out Wed 15 Nov, Due Wed 6 December at 6:00pm [hw10.pdf, Overleaf template, hw10.tex, cos302.cls, Gradescope submission ]
- Assignment 11 -- Out Fri 30 Nov, Due Fri 15 December at 6:00pm (Dean's date, so no extensions possible) [hw11.pdf, Overleaf template, hw11.tex, cos302.cls, Gradescope submission ]

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Can I just watch the old videos rather than come to lecture?**You are responsible for the material I present in lecture, and you should come to lecture. The videos are there to be helpful for review.**Are precepts required?**No, but you should attend precept. Novel material will be presented in them.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.