Princeton University
Computer Science Department

Computer Science 511
Theoretical Machine Learning

Rob Schapire

Spring 2013

 


Directory
General Information | Schedule & Readings | Assignments | Final Project | blackboard

Course summary
Staff, lectures, office hours
Prerequisites
Textbook
Grading and workload
Turning in assignments
Late policy
Collaboration
Scribe notes
     How to be a scribe
Auditors
Communication


Course Summary

Machine learning studies automatic methods for learning to make accurate predictions or useful decisions based on past observations. This course introduces theoretical machine learning, including mathematical models of machine learning, and the design and rigorous analysis of learning algorithms.  Likely topics include: bounds on the number of random examples needed to learn; learning from non-random examples in the on-line learning model; how to boost the accuracy of a weak learning algorithm; support-vector machines; maximum-entropy modeling; portfolio selection.

Here is a tentative list of topics.  (Bullets do not correspond precisely to lectures.)


Staff, lectures, office hours

Lectures:

Tuesday and Thursday, 11-12:20, McCosh Hall, room 4

Professor:

Rob Schapire: 407 CS Building, x8-7726, schapire@cs
Office hours: by appointment.  To schedule an appointment, visit wass.princeton.edu, or send me email if no convenient times are available.

TA:

Zheng (Tracy) Ke: zke@
Office hours: Tuesdays, 4:30-5:30, Sherrerd 107.

Graduate Coordinator:

Melissa Lawson: 310 CS Building, x8-5387, mml@cs


Prerequisites

This class has no formal prerequisites.  However, it is assumed that students will have a general background in computer science (especially theoretical computer science), and some experience with rigorous problem solving and mathematical proof techniques, since these are central to the course.  Background in basic probability is recommended (at least at the level used in COS340 or COS402).  Some calculus and linear algebra and perhaps a tiny bit of analysis will be used.


Textbook

Foundations of Machine Learning
by Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar
MIT Press, 2012

I may also post other papers or notes for topics not in the book.

A copy of this textbook is (or soon will be) on reserve at the Engineering Library.


Grading and workload

Grading will be based on homework assignments (65%) and a final project (35%).  Homeworks will be given roughly every week or two, and will each consist of a small number of problems (no programming).  In all cases, failure to complete any significant component of the course may result in a grade of D or F, regardless of performance on the other components.  Final grades may be adjusted slightly upward for regular and positive class participation, or slightly downward for absence from lecture.

For the final project, you can pick any topic you want for further study.  Your project could involve implementing an algorithm, running experiments, doing further reading, or trying to theoretically extend a result of interest.  In all cases, the project must have a theoretical component, and the end product will be a written report.  More details will be given later in the semester.

You can track how well you are doing using blackboard (log on using your OIT login and password), bearing in mind that these posted grades do not include late penalties or honors optional points.  As a rough guide, if the total number of points you get on your homeworks falls close to or below 65% of the total points possible, then you may be heading for a final grade of D or F, and you should certainly seek assistance.


Turning in assignments

Homeworks can be turned in directly to Tracy immediately before or after class, or you can hand them to her personally in her office (Sherrerd 219), or you can put them in the COS511 mailbox which is located in Sherrerd 123 (this room should be accessible at all times with a Princeton ID).

Please submit hard copy only, and if you are submitting when no one is around to accept your homework, be sure to mark the date and time.


Late policy

All assignments are due at 11:59pm on the due date.

Each student will be allotted seven free days which can be used to turn in homework assignments late without penalty.  For instance, you might choose to turn in HW#1 two days late, HW#4  three days late and HW#6 two days late.  Once your free days are used up, late homeworks will be penalized 20% per day.  (For instance, a homework turned in two days late will receive only 60% credit.)  Homeworks will not receive credit more than five days past the deadline, whether or not free days are being used.  Even so, all homeworks must be completed and turned in, even if this five-day limit has passed.  As noted above, failure to do so may result in a final grade of D or F.

Exceptions to these rules will of course be made for serious illness or other emergency circumstances; in these cases, please contact me as soon as you are aware of the problem.

In counting late days, a weekend (that is, Saturday and Sunday together), count as a single "day".  For instance, a homework that is due on Thursday but turned in on Sunday would be considered two days late, rather than three.

If you are turning in a late homework after hours when no one is around to accept it, please indicate at the top that it is late, and clearly mark the day and time when it was turned in.  Failure to do so may result in me or the TA considering the homework to be submitted at the time when we pick it up (which might be many hours, or even a day or two after when you actually submitted it).

It is your own responsibility to keep track of how many late days you have used.  We will post our own record of late days used on blackboard, but the numbers posted will not always be fully up-to-date.

Note that we do not generally mark late penalties on your returned assignments, but will deduct them from the recorded grade in computing your final grade for the semester.  Knowing how many late days you have used, you can do the same calculation on your own.


Collaboration

Collaboration on the problem sets is allowed in this course, subject to the following guidelines:


Scribe notes

Because there is no perfect textbook for this course, students will be asked to take turns preparing "scribe notes" for posting on the course web site (specifically, on the Schedule and Readings page).  Each class, one student will be the designated "scribe", taking careful notes during class, writing them up, and sending them to me for posting on the web.  Here is more information on how to be a scribe.  I will not grade the scribe notes, but I will give credit for doing them (about equal to one problem set).


Auditors

Auditors are welcome, and are encouraged to take part in class discussions.  If you wish to receive official "credit" for auditing this course, you must attend the vast majority of the lectures.


Communication

An email list is maintained by the university for all students who are registered for the course.  This list will be used by the course staff for important general announcements, such as last minute corrections to the homeworks, changes in due date, etc.  To avoid missing these announcements, it is imperative that you be registered for the course.

This semester, we will be trying out "Piazza", an online forum that will be used for general announcements (in addition to the mailing list), and that can be used by students for discussing course material, general-interest questions about the homeworks, etc.  The course staff will attempt to monitor and respond to questions posted on Piazza.  Students should monitor these discussions since clarifications and corrections of problems may be posted as part of our responses.  Of course, if your question is specific to your own work, you will probably want to contact a staff member directly, and you should always avoid giving away the solution to a problem or programming assignment in a general posting.  To sign up for Piazza, visit:  https://piazza.com/princeton/spring2013/cos511.