Instructor Danqi Chen (danqic AT cs.princeton.edu)
Undergraduate TAs Jacqueline He (jh70 AT princeton.edu)
Jenny Sheng (js112 AT princeton.edu)
Meetings Monday 11:00am-12:20pm
Location COS 402
Office hours Danqi Chen: Monday 2-3pm, Zoom or COS 412 (by appointment)
Jacqueline He: Tuesday 7-8pm (Zoom link)
Jenny Sheng: Wednesday 11am-12pm (Zoom link)

Recent advances in deep learning have ushered in extremely exciting developments in natural language understanding (NLU), which deals with computers’ capability of understanding and comprehension of human languages, one of the most challenging problems in artificial intelligence. This seminar aims to provide students the opportunity to learn the basics and acquire hands-on experience in building and learning about state-of-the-art NLU systems through a semester-long project. We will particularly focus on cutting-edge techniques including pre-trained language models (e.g., BERT, RoBERTa, T5/BART), fine-tuning/prompting, and recently-developed NLU benchmarks. Types of projects include developing better NLU models (more accurate, less compute, less storage, or more data-efficient), adapting existing approaches to new tasks of interest, analyzing and understanding the limitations of current systems, or developing better benchmarks and metrics.

Students are NOT required to have prior experience with NLP (but it is definitely useful) but are strongly encouraged to have taken COS324 before. Students may work in pairs if the project can be split so that each student has their own semester-size piece of the project. We will spend the first 2-3 meetings of the semester surveying the state-of-the-art in the field, brainstorming ideas, and developing project proposals. The remaining meetings will be used for project updates, formal student presentations, and discussions on how to perform background research, prepare a presentation, and write a final paper.

Logistics

  • Make sure you check out the important steps and deadlines at https://www.cs.princeton.edu/ugrad/independent-work/important-steps-and-deadlines. Check out "Getting started: computer science indepdent work".
  • We are mainly going to use Slack for communications this semester (not Ed!). Make sure that you are added to the Slack team in the first week. For important announcements, we will still use emails.
  • Grading: IWs don’t have a clearly defined grading rubric but your grade will depend on the depth and quality of your research work, the quality of your writeup, attendance and participation in the seminar, and meeting all IW deadlines.
  • We are going to maintain a list of useful resources in this google document (will be updated continuously).

Expectations:

Compared to other classes, IWs have a lot of intellectual freedom with very little handholding, while continuing to provide plenty of support if you choose to take advantage of it. We expect you to:
  • Actively participate during the seminar meetings:
    • Show up (on time, every time)
    • Complete the pre-action items
    • Engage and provide thoughtful feedback to other students
    • Help us help you (bring questions, concerns, progress updates)
  • Reach out if you need additional help, in the following order::
    • Ask questions on Slack
    • Attend TA office hours
    • Attend Prof. Chen's office hours (make an appointment first!)
  • Follow all IW deadlines
  • Produce a great project

Schedule

Pre-class action items should be completed by 9am on Mondays.

Week Date Topic Pre-class Action Items
1 January 24 Introduction
Fill out the introduction form
2 January 31 Brainstorming project ideas
Join Slack + fill out the brainstorming form
3 Febuary 7 Project proposal presentations
Upload slides to GD folder
4 Febuary 14 Project updates
Febuary 17 Deadline for a written project proposal (11:59pm)
5 Febuary 21 Project updates
6 Febuary 28 Project updates
March 5 Deadline for the checkpoint form (11:59pm)
7 March 7 Spring Recess (no meeting)
8 March 14 Project updates
9 March 21 Project updates
10 March 28 Project updates
11 April 4 In-class oral presentation
12 April 11 In-class oral presentation
April 13 Deadline for submitting video presentation (11:59pm)
13 April 18 Wrap-up meeting
April 20 Deadline for final report (11:59pm)