Announcement

Sept 4, 2026We have increased capacity to 80 students.
Aug 28, 2026Audit policy: everyone is free to audit as long as it is permitted by school policy and the classroom has space. However, we do not have computing resources for auditing students.
Aug 17, 2026Welcome to CS2680 Modern AI Systems: Agents and System Optimizations! Class meets Monday and Wednesday in SEC LL2.229.

Overview

This course follows one agentic system from the agent loop down through the serving stack. You first use a modern coding agent (Claude Code), then design your own agent using provided API access, measure it, and optimize it. Then you serve an open-weight model yourself and take responsibility for the layers underneath: batching and scheduling, KV-cache and prefix reuse, routing, quantization, and speculative decoding.

Class time and location

Monday and Wednesday 11:15am – 12:30pm SEC LL2.229

Prerequisites

CS61, and at least one graduate-level course in computer systems. Comfort with Python and PyTorch is assumed. See the background we assume.

Assessment

Five assignments and a final project. Class participation throughout the semester. There is no exam.

Textbook. No textbook is required. Background reading:

Computer systems: Operating Systems: Three Easy Pieces
Machine learning systems: Machine Learning Systems
Large language models: Foundations of Large Language Models
Deep learning: The Little Book of Deep Learning

Format. The course runs in two parts.

Part I — Introduction to LLMs and agents. You will learn about how to use coding agents (Claude Code) and how to design an agent. You will also explore various agent architectures, use cases and optimizations.
Part II — Systems for LLMs and agents. We will talk about LLM serving (batching, scheduling, routing, caching, quantization, and speculative decoding) and agentic systems.

The assignments walk down that stack in five steps. First, Mads-Lens¹ lets you watch the agent work (Assignment 1). Second, you build the loop in Mads-Loop (Assignment 2). Third, you make every token count in Mads-Opt (Assignment 3). Fourth, you own the model in Mads-Serve by serving it on the provided GPUs (Assignment 4). Fifth, you optimize the full stack in Mads-Stack (Assignment 5).

¹ “Mads” abbreviates Modern Agent Design and Systems, the arc of this course, and nods to MaDSys, Harvard’s Measurement and Design of Systems research group, which teaches and supports it.

What this course expects. This course will not teach you every piece of knowledge you need for the assignments. It focuses instead on the skills to learn with, and points you in directions worth going. Expect to spend a significant amount of time learning and experimenting on your own after class. If you want an A, plan on it.

Grading.

Assignment 1 Mads-Lens: watch the agent work10%
Assignment 2 Mads-Loop: build the loop10%
Assignment 3 Mads-Opt: make every token count20%
Assignment 4 Mads-Serve: own the model20%
Assignment 5 Mads-Stack: optimize the full stack10%
Project20%
Class participation10%
Bonus optional presentations up to 4% each; course feedback up to 6%; problems current AI cannot solve, 2% eachup to 10%

See policy page for details.

Course website. https://cs2680.com

Staff

Juncheng Yang

Instructor

Yiyu Liu

Advaith Ravishankar

Charles Wang

Yunjia Zheng

Contact: [email protected] Please use this address for all course-related questions.

Office hours:

Instructor: Appointment only
TFs: See Ed + by appointment

Schedule

Tentative and subject to change. Rows with Students as the speaker are student-led: either a paper discussion (see the paper discussion page) or an assignment sharing session. Paper presentations are optional and can earn up to four bonus percentage points each.

Week	Date	Speaker	Topic	Readings	Notes
Part I — Introduction to LLMs and agents
1	2026-09-02	Juncheng Yang	Course overview and logistics [slide]		Assignment 1 out (Mads-Lens: Watch the agent work); Sign up for AWS
2	2026-09-07	No class — Labor Day
2	2026-09-09	Juncheng Yang	Agents use I: How Claude Code Works [slide]	How I use LLMs; Best practices for Claude Code; Agentic Engineering Patterns; more reading →
3	2026-09-14	Juncheng Yang	Agents use II: 100x Engineer with Multi-agents [slide]	Claude Code agent teams; Software is changing (again); Research acceleration: The view inside OpenAI; more reading →
3	2026-09-16	Juncheng Yang	Agents design I: context management and tool design [slide]	ReAct; A practical guide to building agents; LLM Powered Autonomous Agents; more reading →	Assignment 1 due Sun Sep 20, 11:59pm
4	2026-09-21	Students	Student sharing I	What you learned from using an agent (Assignment 1): surprises, failures, lessons, insights, new tools/skills/plugins	Assignment 2 out (Mads-Loop: Build the loop)
4	2026-09-23	Juncheng Yang	Agents design II: multi-agent architecture, coordination, and communication [slide]	multi-agent research system; scaling agent systems; don’t build multi-agents; more reading →
5	2026-09-28	Juncheng Yang	Agents design III: agent memory and reliability [slide]	where agents fail; MemGPT; more reading →
5	2026-09-30	Juncheng Yang	Agent design IV: agent evaluation and cost-effective agent [slide]	Agent evaluation and benchmarks	Assignment 2 due Sun Oct 4, 11:59pm
6	2026-10-05	Harsh Nilesh Pathak (Snowflake)	Guest lecture: Developing and Evaluating Agents in Enterprise Setting		Assignment 3 out (Mads-Opt: Make every token count)
6	2026-10-07	Juncheng Yang	Advanced agent design I: Cost, performance and others		Pre-proposal due (optional and only submit if you want to discuss your ideas)
7	2026-10-12	No class — Indigenous Peoples’ Day
7	2026-10-14	Students	Advanced agent design II: agent safety, self-evolving harness
Part II — Systems for LLMs and agents
8	2026-10-19	Juncheng Yang	Efficient LLM serving: basics I		Assignment 3 due Tue Oct 20, 11:59pm
8	2026-10-21	Juncheng Yang	Efficient LLM serving: basics II		Assignment 4 out (Mads-Serve: Own the model)
9	2026-10-26	Juncheng Yang and Students	Assignment Feedback and Student sharing II	Common pitfalls in your submission; Assignment 3 lessons sharing.
9	2026-10-28	Juncheng Yang	Efficient LLM serving: speculative decoding		Project proposal due
10	2026-11-02	Juncheng Yang	Efficient agentic systems: scheduling and prefix caching		This lecture is earlier because of Assignment 4.
10	2026-11-04	Juncheng Yang	Efficient LLM serving: pruning and quantization
11	2026-11-09	Xianzheng Kou (Meta)	Guest lecture: Why Infrastructure sets the pace of AI		Assignment 4 due Tue Nov 10, 11:59pm
11	2026-11-11	Juncheng Yang	Efficient LLM serving: GPU and kernels		Assignment 5 out (Mads-Stack: Optimize the full stack)
12	2026-11-16	Students	Efficient LLM serving: KV-cache optimization and model architectures
12	2026-11-18	Students	Student sharing III	Demonstrate your agent-engine co-design (Assignment 4)
13	2026-11-23	Juncheng Yang	Efficient agentic system: routing and load balancing		Assignment 5 due Sun Nov 29, 11:59pm
13	2026-11-25	No class — Thanksgiving recess
14	2026-11-30	Juncheng Yang	Efficient agentic systems: others
14	2026-12-02	Students	Student sharing IV	Showcase your full stack optimizations (Assignment 5)
14	2026-12-07	Students	Final project poster and demo	Present your final project as a poster and a live demo. Location: SEC 1.402 from 9:00 AM to 1:00 PM	Final project due Tue Dec 8, 11:59pm

Reference pages

Agent evaluation and benchmarks. Evaluation design, metrics, reliability, cost, and benchmark selection.

Compute resources

Setup instructions for every environment the course uses are on the computing setup page. Compute for the assignments and the final project comes from: