Statistics 300: Advanced topics in Statistics:
Bayesian nonparametrics

overview

This course is an introduction to Bayesian nonparametrics. Nonparametric Bayesian methods provide a principled, data-driven way of dealing with questions of model complexity. Some models (and some of their typical uses) that we will cover in this course are:

Dirichlet processes (which can automatically choose an appropriate number of clusters for a mixture model)
Hierarchical Dirichlet processes (which can automatically choose an appropriate number of "topic" clusters in a mixed-membership mixture model, or an appropriate number of states in a hidden Markov model)
Beta processes (which can automatically choose an appropriate number of factors for a factor analysis model)
Gaussian processes (which can learn arbitrarily complex mappings from inputs to outputs)

We will also discuss some applications of these models, and the computational methods (chiefly Markov chain Monte Carlo) that make it possible to actually apply them to real data analysis problems.

logistics

The instructor is Matt Hoffman.
Class meets Wednesdays from 1:15 pm to 4:05 pm in building 540, room 103.
Matt's office hours will be in Sequoia Hall, room 206, Wednesdays from 11 am to 1 pm.
The TA is Yunjin Choi. Her office hours will be in Sequoia Hall, room 208, Thursdays from 10 am to 12 pm.
There is a course email list for this class. Please sign up for it if you haven't already.

evaluation

There will be two assignments, each of which will count for 25% of the final grade. The remaining 50% of the final grade will be based on class participation.

tentative schedule, readings, and scribe notes

June 26	Introductions, review of Bayesian statistics, review of the Dirichlet distribution, introduction to the Dirichlet process (DP) and DP mixtures.	No reading.	Scribe notes.
July 3	Review of Markov chain Monte Carlo (MCMC), MCMC for DP mixtures.	Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford Neal. (Original tech report version available here.) Optional review of MCMC methods: Probabilistic Inference Using Markov Chain Monte Carlo Methods by Radford Neal.	Scribe notes part 1 and part 2.
July 10	Topic models, the hierarchical Dirichlet process (HDP).	Hierarchical Dirichlet Processes by Teh, Jordan, Beal, and Blei. Latent Dirichlet Allocation by Blei, Ng, and Jordan.	Scribe notes.
July 17	HDPs continued.		Scribe notes part 1 and part 2.
July 24	Guest lecture, David Knowles.
July 31	HDP-based hidden Markov models, sticky HDP-HMMs, variational inference.	A Sticky HDP-HMM with Application to Speaker Diarization by Fox, Sudderth, Jordan, and Willsky. An Introduction to Variational Methods for Graphical Models by Jordan, Ghahramani, Jaakkola, and Saul.	Scribe notes.
August 7	Gaussian processes, Beta processes.	Gaussian Processes for Machine Learning by Rasmussen and Williams. The Indian Buffet Process: An Introduction and Review by Griffiths and Ghahramani.	Scribe notes.
August 14	Qualifying exams.

Note that this schedule is subject to change.

assignments

Assignment 1 is now available, and is due on Friday, August 2.

Statistics 300: Advanced topics in Statistics: Bayesian nonparametrics