COS IW 03: Hands-on Reinforcement Learning
[Logistics]   [Resources]   [Schedule]




Xi Chen

Xi Chen


How to contact us: Please use Piazza for all questions related to this IW seminar and to find announcements. For external queries, emergencies, or personal matters, you can use a private Piazza post visible only to Instructors.

What is this seminar about?

Reinforcement learning has enjoyed a healthy resurgence in recent years, with super-human performance being demonstrated on domains like Go, Starcraft and Atari. This seminar aims to provide students the opportunity to learn the basics and acquire hands-on experience with RL techniques through a semester-long project. Types of projects include benchmarking state-of-the-art RL algorithms on various tasks, proposing new algorithms for more sample-efficient or generalizable RL, applying existing techniques to solve problems in other domains (e.g. computer vision, NLP, program synthesis, or even job scheduling!), or development of new evaluation frameworks, tasks and metrics for RL.
There are no prerequisites for this seminar beyond COS 217, COS 226 and one of COS 324 (Machine Learning), 484 (NLP), 485 (Neural Networks) or a similar machine learning course. You will be expected to implement RL algorithms, preferably using standard libraries like PyTorch, Tensorflow, OpenAI gym, etc. Students may work in pairs IF the project can be split so that each student has their own semester-size piece of the project. We will spend the first 1-2 meetings of the semester surveying standard and current RL algorithms, brainstorming ideas and developing project proposals. The remaining meetings will be used for project updates, formal student presentations, and discussions on how to perform background research, prepare a presentation and write a final paper.



Some helpful books to read up on background material:

Useful web resources:



Pre-class action items should be completed by 11:59pm on Mondays before class on Tuesdays. Please also keep track of general IW events and deadlines here.

Date Topic Pre-class Action items
Feb 4 Introduction Fill out this form
Feb 6 Information meeting for all IW students (CS 105, 4:30-5:30pm)
Feb 7 Introduction to PyTorch
Feb 11 Brainstorming project ideas Project ideas form
Feb 18 Project proposal presentations Presentation in Google slides
Feb 25 Project updates Update Google slides
Mar 3 Project updates Update Google slides
Mar 8 Submit checkpoint form
Mar 10 Project updates Update Google slides
Mar 17 No class (Spring recess)
Mar 24 Project updates Update Google slides
Mar 31 Project updates, Attend "How to Give an IW talk" in CS 105 at 4:30pm Update Google slides
Apr 7 Project updates Update Google slides
Apr 14 Project updates, Attend "How to write an IW paper" in CS 105 at 4:30pm Update Google slides
Apr 26 Submit slides for Oral Presentation
Apr 28 In-class oral presentations
May 12 Written final report due