This page collects what is not linked from the main navigation, so that everything published on the site stays reachable. Some entries are pages that may be superseded, draft, or simply not part of the primary reading path; the optional content below lives here in full, because no meeting on the schedule covers it.
The topics below are not covered in lecture. None of them has a class meeting or a paper discussion, and nothing on the schedule assumes you worked through them.
Working with agents in practice. Specification, cheap verification, and when a second agent earns its bill. It was on the schedule and came back off it; the lecture notes are written and stay up, so this is a complete meeting you can work through on your own. Assignments 1 and 2 are the place the material actually gets used. The readings page collects the papers with a note on each.
LLMs for systems. Turning the relationship around: models that write, optimize, and evaluate systems code, and the harnesses that decide whether the answer was any good. It ran as Part III on an earlier version of this schedule and came off it when the semester ran out of meetings — the readings are complete, the second meeting's lecture notes are written and stay up, and the material is the premise behind Assignment 5 and a good deal of what a final project tool has to do. The readings page collects the papers with a note on each.
The training side of the stack. How a model gets built, rather than how it gets served. Work through it if you want the background on distributed training, fine-tuning, or checkpointing. The Ultra-Scale Playbook is the best single entry point; the papers below go deeper, and the readings page collects them in one place with more on each.
| Topic | Materials |
|---|---|
| Working with agents: specification, verification, and multi-agent workflows | lecture notes; SWE-agent, Reflexion, AutoGen, KernelBench |
| LLMs for systems research I: agents that write and optimize systems code | SWE-agent, KernelBench; then LLMs for compiler optimization, OpenHands, SWE-bench Multimodal, BountyBench, measure one level deeper |
| LLMs for systems research II: automating the research loop | lecture notes; MLGym, AI Scientist-v2; then automated statistical model discovery, MLE-bench, SWE-bench, OSWorld |
| Distributed training and fine-tuning I: parallelism and ZeRO | start with Megatron-LM, ZeRO; then FSDP, GPipe, PipeDream, Alpa, LoRA, QLoRA, TorchTitan |
| Distributed training and fine-tuning II: memory, failures, and scale | start with Llama 3 (§3, infrastructure); then MegaScale, DeepSeek-V3, ByteCheckpoint, FP8-LM, ZeRO-Infinity, sublinear memory |
| Data pipelines and checkpointing | Data-Juicer, Dolma, DataComp-LM, CheckFreq, GEMINI, 3FS |