Princeton University, Spring 2020

Lecture: Mon/Wed 12:30-1:20pmLecture Location: Friend Center 101

Contact: cos302-s20@lists.cs.princeton.edu

Course Google Calendar

**All office hours are now on Zoom. See Piazza for links.**

Prof. Ryan Adams (OH: Mon/Wed 1-3pm)

TA: Haochen Li (OH: Mon 7-9pm)

TA: Sulin Liu (OH: Thu 7-9pm)

TA: Geoffrey Roeder (OH: Wed 4:30-6:30pm)

TA: Ari Seff (OH: Thu 5-7pm)

TA: Alexander Strzalkowski (OH: Tue 5-7pm)

TA: Fangyin Wei (OH: Fri 4-6pm)

Lab TA: Michael Li (OH: Sun 2-4pm)

Lab TA: Alan Chung (OH: Sat 11am-1pm)

Lab TA: Kenny Peng (OH: Mon 5-7pm, COS 401)

- 5 May 2020: The practice final is available.
- 29 April 2020: Assignment 11 posted.
- 21 April 2020: Assignment 10 posted.
- 21 April 2020: Assignment 9 posted.
- 15 April 2020: Assignment 9 posted.
- 7 April 2020: Assignment 8 posted.
- 30 March 2020: Assignment 7 posted.
- 23 March 2020: Assignment 6 posted.
- 23 March 2020: Due to the disruption and the need for more dynamic communication, please keep an eye on Piazza for links to Zoom, lecture videos, etc.
- 11 March 2020: The midterm deadline has been extended to
**6:00pm ET, Friday 20 March**due to the disruption from COVID-19. - 9 March 2020: The midterm exam will now be a
**take home exam**:- The exam will become available as a PDF on Blackboard at 12:30pm on Wednesday 11 March.
- You will print a hard copy of the exam and write your answers on the hard copy.
- If you are unable to print it for some reason, email the staff list for assistance.
- You must complete the exam in a two-hour window before 12:30pm on Friday 13 March.
- You will write your start and end times on the exam.
- No calculator, phone, or other electronic device is allowed.
- After you complete the exam, you will scan it to PDF and turn it in via Gradescope.
- We will also ask you to scan your page of notes along with the exam.
- If you are unable to scan the exam or use a scanning app on your phone, we will provide information on how to get assistance from COS administrative staff on how to get it scanned.
- I don't have a scanning app recommendation, but I tend to trust Wirecutter's reviews.
- It is a closed-book, closed-note, no-Internet exam. You may use one double-sided page of handwritten or typed notes.
- The basic content of the exam is the same as it was when you were preparing for an in-class midterm.

- 9 March 2020: Due to COVID-19 we are moving to remote instruction. We'll be putting lecture/precept videos on line and holding office hours via Zoom.
- 8 March 2020: Solutions to midterm practice questions are available.
- 7 March 2020: Assignment 4 solutions available.
- 6 March 2020: Assignment 5 is now posted.
- 4 March 2020: Midterm information:
- In-class at regular lecture time (12:30-1:20pm) on Weds 11 March in Friend 101
- You can bring a double-sided page of notes. They can be handwritten or typed.
- No book or calculator is allowed.
- You'll write on the exam itself. No separate exam book.
- If you miss for medical reasons, you must provide a doctor's note.
- We'll have two review sessions, both in Friend 101: Sun 8 March 2-4pm and Mon 9 March 7-9pm
- Solutions to the midterm practice questions will become available during the Sunday review session.

- 2 March 2020: Midterm practice questions available.
- 1 March 2020: Solutions available now for assignments 1, 2, and 3.
- 28 February 2020: We've been informed that we can't hold office hours in COS 201, so see new OH locations above.
- 24 February 2020: Assignment 4 is now posted.
- 17 February 2020: Assignment 3 is now posted.
- 10 February 2020: Assignment 2 is now posted.
- 3 February 2020: Assignment 1 is now posted.
- 3 February 2020: Watch the screencast on how to get started with Colab.
- 3 February 2020: Watch the screencast on how to get started with Overleaf for your homework.
- 3 February 2020: If you are not able to attend lecture or precept due to University guidelines on the Coronavirus, please email course staff ASAP.
- 3 February 2020: Please fill out the anonymous survey to help us understand your background.
- 3 February 2020: Sign up on the Piazza discussion site.

Mon 3 February 2020
## Topics:

Lecture: Course Introduction [slides]

Assignment 1 Out

- Course goals
- Relationship to other Princeton courses
- Logistics
- Big picture: mathematical tools for finding the best fit

Lecture: Vectors and Matrices

- Vectors
- Systems of linear equations as matrices
- Adding and multiplying matrices
- Matrix transpose and symmetry
- Color vectors notebook
- Sound vectors notebook, beats.tgz (you'll need to mess around with paths a bit probably)
- Face vectors notebook, faces.tgz (you'll need to mess with paths and also point it to the opencv xml file in your local install)

- [required] MML 2.0-2.2
- [optional] 3Blue1Brown video on vectors

Thu/Fri 6/7 February 2020

Precept: Solving linear systems [slides] [COS_302_Precept_1.ipynb]

- Particular and general solution
- Transformations
- Row echelon form
- Gaussian elimination

- [required] MML 2.3

Mon 10 February 2020

Lecture: Solving linear systems, continued [Colab Notebook on Gauss-Jordan]

Assignment 2 Out

- Computing the inverse
- Overconstrained and underconstrained systems
- Applications
- Brief tour of more advanced methods: Jacobi, Gauss-Seidel, Krylov subspaces

- [required] MML 2.3

Wed 12 February 2020

Lecture: Groups and vector spaces [Colab Notebook for Color Subspaces]

Assignment 1 Due

- Groups
- Vector spaces
- Vector subspaces

- [required] MML 2.4

Thu/Fri 13/14 February 2020

Precept: Basis concepts [slides]

- Linear independence
- Rank
- Basis sets

- [required] MML 2.5-2.6
- [optional] 3Blue1Brown video on basis vectors

Mon 17 February 2020

Lecture: Basis concepts, continued

Assignment 3 Out

- Linear maps
- Change of basis

- [required] MML 2.7-2.8
- [optional] 3Blue1Brown video on change of basis
- [optional] 3Blue1Brown video on linear transformations

Wed 19 February 2020

Lecture: Norms and inner products

Assignment 2 Due

- Image and kernel
- Vector norms
- Inner products
- Positive definite matrices

- [required] MML 3.0-3.3

Thu/Fri 20/21 February 2020

Precept: Orthogonality [slides]

- Orthogonal vectors
- Orthogonal bases
- Orthogonal complement
- Orthogonal projections

- [required] MML 3.4-3.8

Mon 24 February 2020

Lecture: Projections, continued

Assignment 4 Out

- Gram-Schmidt orthogonalization

- [required] MML 3.8-3.9

Wed 26 February 2020

Lecture: Eigenvectors and eigenvalues [Colab notebook demo]

Assignment 3 Due

- Determinant and trace
- Eigenvalues and eigenvectors

- [required] MML 4.0-4.2
- [optional] 3Blue1Brown video on determinants
- [optional] 3Blue1Brown video on eigenvectors and eigenvalues

Thu/Fri 27/28 February 2020

Precept: Eigendecomposition and Cholesky factors
[slides, Python notebook]

- Eigenvalue decomposition
- Cholesky factorization

- [required] MML 4.3-4.4

Mon 2 March 2020

Lecture: Singular value decomposition

Assignment 5 Out

- SVD intuition
- SVD computation

- [required] MML 4.5

Wed 4 March 2020

Lecture: Singular value decomposition, continued

Assignment 4 Due

- Approximation of matrices
- Applications of SVD

- [required] MML 4.6-4.7

Thu/Fri 5/6 March 2020

Precept: Other matrix decompositions
[slides]

- QR factorization
- LU factorization

Mon 9 March 2020

Lecture: Catchup and Review [ video 1 on HW4 P2, video 2 on HW4 P5 ]

Wed 11 March 2020

**IN-CLASS MIDTERM EXAM**

Thu/Fri 12/13 March 2020

No precept

Lecture: Differentiation

Assignment 6 Out

- [required] MML 5.0-5.1

Wed 25 March 2020

Lecture: Multivariate differentiation

Assignment 5 Due

- [required] MML 5.2-5.5
- [optional] MML 5.6-5.8

Thu/Fri 26/27 March 2020

Precept: Multivariate differentiation, continued [slides, video]

- Differentiation with respect to matrices
- Useful identities

Lecture: Random variables

Assignment 7 Out

- [required] MML 6.0-6.2

Wed 1 April 2020

Lecture: Random variables

Assignment 6 Due

- [required] MML 6.3-6.4

Thu/Fri 2/3 April 2020

Precept: Sampling from distributions [video, annotated slides, Colab notebook]

- Pseudo-random number generation
- Inverse transform sampling

Mon 6 April 2020

Lecture: Independent and dependent random variables

Assignment 8 Out

Wed 8 April 2020

Lecture: Aggregating random variables

Assignment 7 Due

Thu/Fri 9/10 April 2020

Precept: Transforming random variables [slides, video]

- Transformations
- Change of variables
- Univariate Gaussian distributions

- [required] MML 6.7

Mon 13 April 2020

Lecture: Multivariate Gaussian distributions

Assignment 9 Out

- [required] MML 6.5

Wed 15 April 2020

Lecture: Monte Carlo estimation

Assignment 8 Due

- Integrals as expectations
- Proving unbiasedness
- Variance of Monte Carlo estimators

Thu/Fri 16/17 April 2020

Precept: Monte Carlo estimation, continued [video]

- Rejection sampling
- Importance sampling

Mon 20 April 2020

Lecture: Information theory

Assignment 10 Out

Lecture: Optimization basics

Assignment 9 Due

- [required] MML 7.0-7.1

Thu/Fri 23/24 April 2020

Precept: Constrained optimization

- Constrained optimization
- Lagrange multipliers

- [required] MML 7.2

Mon 27 April 2020

Lecture: Convex optimization

Assignment 11 Out

- Convex optimization
- Linear programming

- [required] MML 7.3

Thu/Fri 30/1 April/May 2020

Precept: Questions and Answers

- Assignment 1 -- Out Mon 3 Feb, Due Weds 12 Feb at 23:55 [hw1.pdf, Overleaf template, hw1.tex, cos302.cls, Gradescope submission, hw1-solutions.pdf]
- Assignment 2 -- Out Mon 10 Feb, Due Weds 19 Feb at 23:55 [hw2.pdf, Overleaf template, hw2.tex, cos302.cls, Gradescope submission, hw2-solutions.pdf]
- Assignment 3 -- Out Mon 17 Feb, Due Weds 26 Feb at 23:55 [hw3.pdf, Overleaf template, hw3.tex, cos302.cls, Gradescope submission, hw3-solutions.pdf]
- Assignment 4 -- Out Mon 24 Feb, Due Weds 4 Mar at 23:55 [hw4.pdf, Overleaf template, mnist2000.pkl cos302.cls, Gradescope submission, hw4-solutions.pdf]
- Assignment 5 -- Out Mon 6 Mar, Due Weds 25 Mar at 23:55 [hw5.pdf, Overleaf template, cos302.cls, dog_names1000.txt, nyt.pkl.gz, Gradescope submission, hw5-solutions.pdf]
- Assignment 6 -- Out Mon 23 Mar, Due Weds 1 Apr at 23:55 [hw6.pdf, hw6.tex, Overleaf template, cos302.cls, Gradescope submission, hw6-solutions.pdf]
- Assignment 7 -- Out Mon 30 Mar, Due Weds 8 Apr at 23:55 [hw7.pdf, hw7.tex, Overleaf template, cos302.cls, Gradescope submission, hw7-solutions.pdf]
- Assignment 8 -- Out Mon 6 Apr, Due Weds 15 Apr at 23:55 [hw8.pdf, hw8.tex, Overleaf template, cos302.cls, Gradescope submission, hw8-solutions.pdf]
- Assignment 9 -- Out Mon 13 Apr, Due Weds 22 Apr at 23:55 [hw9.pdf, hw9.tex, Overleaf template, cos302.cls, Gradescope submission, hw9-solutions.pdf]
- Assignment 10 -- Out Mon 20 Apr, Due Weds 29 Apr at 23:55 [hw10.pdf, hw10.tex, mnist_full.pkl.gz, shakespeare.txt, Overleaf template, cos302.cls, Gradescope submission]
- Assignment 11 -- Out Mon 27 Apr, Due Tues 12 May at 23:55 [hw11.pdf, hw11.tex, Overleaf template, cos302.cls, Gradescope submission,

- Assignments: 60% (lowest dropped)
- Midterm Exam: 20%
- Final Exam: 20%

**Does this course count towards****the SML certificate as a "Foundations of ML"?**No it does not. This is not a machine learning course in of itself. This course is intended to help you get the background to take machine learning and other courses that require continuous mathematics.**Does this course count towards the COS applications track?**No.**Can I take this concurrently with MAT 202?**Yes.**Are undergraduates and graduate students graded the same way?**Everyone's assignments and exams are graded with the same rubric, but the final grades will be curved separately.**Are precepts required?**You should attend precept. There will be new material presented in precept each week, and the assignments and exams will assume that knowledge.**Why don't you do late days?**It introduces substantial additional bureaucracy and bookkeeping. I would rather that energy be spent working with students and making the course better.**Will this course be offered every semester?**Yes, that is the plan starting from spring 2020.**What is Metacademy?**Metacademy is an exciting online tool developed by Roger Grosse and Colorado Reed for helping you to develop personalized instruction. It's meant to help you manage what you know about different topics and develop an individualized curriculum to learn a new subject.