Course Schedule

Please note that this schedule is subject to change.

Lecture 1: Course introduction and overview

September 4, 2024

Slides (pdf)

Topic 1: LLM Serving - core systems decisions

Lecture 2

September 11, 2024

Orca: A Distributed Serving System for Transformer-Based Generative Models, OSDI 2022
Research project team formation due by 5pm ET

Lecture 3

September 18, 2024

Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve, OSDI 2024

Lecture 4

September 25, 2024

Efficient Memory Management for Large Language Model Serving with PagedAttention, SOSP 2023

Topic 2: Local LLM Serving with Consumer-Grade GPUs

Lecture 5

October 2, 2024

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU, SOSP 2024

Topic 3: Compound AI systems: RAGs and Agents

Lecture 6

October 9, 2024

Parrot: Efficient Serving of LLM-based Applications with Semantic Variable, OSDI 2024
Research project proposal due by Friday, 10/11 at 5pm ET

Fall Recess

October 16, 2024

Lecture 7

October 23, 2024

Lecture 8

October 30, 2024

INFERCEPT: Efficient Intercept Support for Augmented Large Language Model Inference, ICML 2024

Topic 4: Mixture of Experts

Lecture 9

November 6, 2024

Topic 5: Low-rank adaptation (LoRA)

Lecture 10

November 13, 2024

dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving, OSDI 2024

Topic 6: Image generation

Lecture 11

November 20, 2024

Thanksgiving Recess

November 27, 2024

Class Research Presentations

December 4, 2024