6 - Learning and Optimization
March 2nd, 2010.
Slides
Scribe Notes
N/A. 
Summary
- Comments about homework.
- Taxonomy of Optimization
- Line search algorithms
- Simple Gradient descent algorithms
- The Hessian matrix and its uses
- Stochastic gradient algorithms
Recommended books
Two very different books:
- P. G. Ciarlet: Introduction to Numerical Linear Algebra and Optimization. Cambridge University Press, 1989.
- Yu. Nesterov: Introductory Lectures On Convex Optimization. Springer-Verlag, 2003.
