agent run --goal "summarize last week's meeting notes"
- contextsystem + user + notes_search
- tokenizeuser → [141249, 277, 750, …]
- prefillKV cache ready
- decodenotes_search({query: "last week's meetings"})"3 decisions, 2 action items"
- tool4 notes + excerpts
- feedbackmessages += [tool_call, tool_result]
- respondstream → user · trace ✓
The roadmap
Section titled “The roadmap”Topic jumps to that section of the chapterM Model project (optional exercises)A System project (optional exercises) Security checkpointHow to read S/A/M codes
Start here
Part I ModelsOptional · deep route
Take it when you want to know how models work; if you only build applications, skip ahead to S6 LLM APIs and Application Basics.
The 5 chapters of Part I (S1–S5); open this or switch to Deep mode to show them
- How do you teach a model to answer?
- How does LoRA save memory?
- How do you train a model with a score?
- How do you teach without a right answer?
- How are reasoning models trained?
- Can you get more accuracy without changing the model?
- Why do models game the reward?
- How does a small model learn from a big one?
- How do you know fine-tuning worked?
Part II Applications
- How does the system actually fail?
- Where do the test cases come from?
- Who decides pass or fail, and how?
- Can you trust a model grader?
- How big a gap counts as a real improvement?
- Measure capability or reliability?
- Can you trust public benchmarks?
- Does the model know when it’s unsure?
- What does one task cost?
Part III Knowledge & Tools
- How do documents become searchable text?
- How big should chunks be?
- How does embedding search find similar passages?
- How do keyword and vector search work together?
- Should you rewrite the query before searching?
- When an answer is wrong, which stage broke?
- How do you measure retrieval and generation separately?
- Why not put everything in the context?
- Prompt, RAG or fine-tuning?
Part IV Agents
- What does agent quality cover?
- Grade the outcome or the steps?
- How do you score results that vary every run?
- How do you make every trial fair?
- Can machine settings change the score?
- Does a higher score always mean better?
- What extra pitfalls do agent benchmarks have?
- What if there are too many traces to read?
- Can a better prompt be found automatically?
- Can routing go to a cheap small model?
- Is this harness component worth keeping?
- When is it worth splitting into multiple agents?
- How do multiple agents divide the work?
- What should a handoff carry?
- Who splits the work and merges the results?
- How much more does multi-agent cost?
- How does multi-agent usually fail?
- Is multi-agent worth keeping?
- How do parallel edits avoid overwriting each other?
Part V Production
- Do injection attacks really happen?
- How do you find the dangerous combination?
- Why can’t detection stop it?
- How do you make injection harmless?
- How much damage can a hijacked agent do?
- Can you trust your plugins?
- How do you set the guardrail threshold?
- Where does the data live, and who regulates it?
- Will the model follow its spec?
- Should AI handle this at all?
- How do you make waiting bearable?
- How should tool results be displayed?
- How do you handle stop, errors and empty results?
- How do you make AI mistakes easy to undo?
- How much should users trust it?
- When should you ask the user?
- How do you collect user feedback?
- How do you keep proactive notifications from being annoying?
- How do you know the interface is usable?
- How do you keep users from seeing each other’s data?
- How do you keep the bill under control?
- Which model should you start with?
- Which model should each request go to?
- Self-host or use an API?
- How many users can the system handle?
- What if a long task fails midway?
- How do you ship a prompt change safely?
- How do you notice quality getting worse?
Part VI Capstone
Choose a route
Section titled “Choose a route”Application route (for most people)You want to build AI apps or agents: S0 AI Systems Overview → Model Crash Course → Part II to Part VI (S6–S19). No machine learning background needed.
Concepts route (no coding)You want to understand how AI systems work: read the lectures and do each chapter's Try it first and self-check.
Full route (including how models work)You also want to know what happens inside the model: go from S0 AI Systems Overview through S19 Real Agent Systems and the Capstone in order. Part I Models (S1–S5) is lower-level and uses more math.
You can follow the concepts route without programming: read the lectures, Try it first and self-check sections to understand how AI systems work. The hands-on routes require basic programming, HTTP and JSON.
Choose a reading mode at the top of each page. Simple shows the main explanation for a first read. Deep also opens implementation details, derivations, optional topics and advanced exercises.
What the system project leaves you with
Section titled “What the system project leaves you with”The advanced exercises build one system across chapters. By the end, you have a deployed agent you use daily, with RAG, tools, memory, evals and security guardrails, plus engineering evidence others can review. Each step adds a capability. The number after A matches its chapter; see How to read the codes:
| Stage | What your system gains |
|---|---|
| Part II Applications (A6–A8) | A streaming chat UI (Gradio, Chainlit or a CLI work too) with traces, reliable structured output and workflows, and a golden set and judge grown from real traces |
| Part III Knowledge & Tools (A9–A11) | RAG with citations, tools and an MCP server with risk levels, and a context builder with a budget and caching |
| Part IV Agents (A12–A15) | A hand-written harness, layered memory, trajectory evals and prompt optimization; multi-agent stays only if it wins the eval |
| Part V Production (A16–A18) | A security audit and fixes, a complete frontend, and a multi-user deployment with an eval-gated release process |
| Part VI Capstone (A19) | The capstone: extend it into an always-on agent |
The full breakdown is in System Project A6–A19.
Two hands-on projects
Section titled “Two hands-on projects”Model project M1–M5From autograd to your own tiny GPT, then SFT, DPO and GRPO on a small model, and finally a mini inference server.
System project A6–A19Pick an application you care about and grow it from a streaming chat into a deployed agent with RAG, tools, a harness, memory, evals and security guardrails.
Lecture content is licensed under CC BY-NC-SA 4.0: you may share and adapt it with attribution, not for commercial use, and adaptations must use the same license. Quoted third-party content and trademarks belong to their owners.