Intermediate

Attention is the mechanism that made modern language models possible. The problem it solves: to represent a word properly you need to know which other words it relates to. In "the server rejected the request because it was stale", the token for it is meaningless until you know it refers to the request.

Attention handles that by having every token score every other token, producing a weight for how much to read from each. Each token is then rebuilt as a weighted blend of the others, so it absorbs the context it needs. Stack that operation dozens of times with different learned weights, and you get a transformer.

• ArticleThe Illustrated Transformer Jay Alammar. Still the clearest written walkthrough, diagram by diagram.
• PaperAttention Is All You Need Vaswani et al., 2017. Read it once you have watched the videos. Section 3.2 is the part that matters.
• VideoGradient descent, how neural networks learn 3Blue1Brown. Watch this and Backpropagation if training is still a black box to you.

There is a point at which reading about transformers stops paying and typing one out starts. Karpathy builds a working character-level language model from an empty file in two hours, explaining every line. Do it by typing along, not by watching.

What you get from this is not the ability to train production models. It is that the architecture stops being a diagram and becomes code you have debugged, which changes how you reason about everything downstream.

• CourseNeural Networks: Zero to Hero The full series with notebooks, in order.

An embedding converts a piece of text into a fixed-length list of numbers positioned so that things with similar meaning sit near each other. "Cancel my subscription" and "how do I unsubscribe" end up close together despite sharing one word. A vector database stores millions of those and finds the nearest ones fast.

The thing to understand is what "near" means, because it is nearness in meaning as judged by the embedding model, and that model has its own opinions. Negation often barely moves a vector, so "the deploy succeeded" and "the deploy did not succeed" can look similar. Numbers, dates and identifiers embed badly. These are not edge cases, they are Tuesday.

The beginner version of RAG works in a demo and disappoints in production. The gap is almost never the model. It is chunking, hybrid search and reranking, in that order of impact.

• Chunking. Splitting every 500 characters cuts sentences in half and separates a heading from the rule underneath it. Split on structure: headings, sections, function boundaries. Keep the heading with the body.
• Hybrid search. Vectors are bad at exact terms. Someone searching for an error code or a product name wants a keyword match. Run both keyword and vector search and merge the results.
• Reranking. Retrieve twenty candidates cheaply, then have a smaller model score each one against the question and keep the best four. This is usually the single largest quality jump available.
• Give the model the source. Pass the document name and section with each chunk so it can cite, and so you can tell where a wrong answer came from.

• ArticleIntroducing contextual retrieval Anthropic. Prepending a short generated summary to each chunk before embedding, and what it does to failure rates.
• CourseRAG evaluation Hugging Face cookbook. How to score a retrieval pipeline instead of guessing.

This question comes up in every project and the answer is usually the cheapest option on the list. Work down it in order and stop as soon as something works.

A tool is a function you describe to the model in words, so that it can ask for it by name with arguments. The description is the interface, and writing it is a genuine design task: the model has your description and nothing else, no source to read and no colleague to ask.

The Model Context Protocol is an open standard for exposing tools and data to any AI application, so you write a server once and every MCP-aware client can use it. It is the reason a tool you build for Claude Code also works elsewhere.

• DocsModel Context Protocol The specification and quickstarts for building servers and clients.
• GuideWriting effective tools for agents Anthropic. Read this before you write your second tool.
• DocsMCP in Claude Code Connecting a server to your own sessions.

Prompt engineering asks what to write. Context engineering asks what should occupy the window on this call, and it has to be answered again on every single turn as files are read, tools return and the conversation grows. It is the discipline that replaces prompt engineering once a system has more than one turn.

The goal is the smallest set of high-signal tokens that gets the outcome you want. That is harder than it sounds, because the instinct is always to add: one more instruction, one more example, the whole file instead of the function. Every addition dilutes everything else.

• GuideEffective context engineering for AI agents Anthropic. The core reference for this section. Read it twice.
• VideoContext engineering vs prompt engineering IBM Technology, 7:52. Useful if the distinction still feels fuzzy.

You met the loop on the beginner track. Here you build one and meet its real problems: it calls the same tool forever, it declares success without checking, it burns the context window on tool output, and it fails on step fourteen with no record of steps one to thirteen.

None of those are model problems and none are fixed by a better prompt. They are fixed by the code around the model, which is why the advanced track is mostly about that code.

• GuideBuilding effective agents Anthropic. Prompt chaining, routing, parallelisation, orchestrator-workers, evaluator-optimizer. Learn these names, they are the shared vocabulary.
• CourseHugging Face AI Agents Course Free, with a certificate. Covers smolagents, LlamaIndex and LangGraph, plus observability and evaluation.
• PaperReAct: Synergizing Reasoning and Acting The reason the loop is shaped the way it is.

On the beginner track you used Claude Code. Here you configure it, which means treating it as a harness you shape rather than a chat window you talk to. Four mechanisms, roughly in the order they pay off.

• DocsBest practices Patterns for getting real work out of it.
• DocsSkills and hooks The two features most teams underuse.
• TalkHow Anthropic uses Claude Code NDC, one hour. How the team that builds it actually works.

Vector search finds passages that resemble your question. It cannot answer a question about how things connect, because the connection is not in any single passage. That is what a graph is for, and the two are complements rather than competitors.

• CourseNeo4j GraphAcademy Free. Graph Data Modeling Fundamentals, then Building Knowledge Graphs with LLMs.

A benchmark is a fixed set of tasks with known answers, used to compare two things. An eval is the same idea applied to your own system, and it is the difference between engineering and guessing. Without one, every change is a vibe and every regression is discovered by a user.

Start embarrassingly small. Twenty real cases with known good answers, in a file, scored by a script you can run in a minute. Twenty cases you actually run beats two hundred you plan to write.

• GuideDemystifying evals for AI agents Anthropic. The most practical thing written on this.
• LeaderboardSWE-bench Real GitHub issues, judged by whether the test suite passes. The standard for coding agents.
• LeaderboardArena Formerly LMArena. Blind human preference between two models. Measures what people like, which is not the same as what is correct.
• LeaderboardTerminal-Bench Agents doing real terminal work.

• You can explain attention without using the word attention.
• You have a retrieval pipeline over our documents with a measured hit rate, and you know which chunking change moved it.
• You have written an MCP tool that someone else on the team has used.
• You can name the four context failure modes and point at one in a log.
• You have built a three-tool agent with a step ceiling, and read its log end to end.
• You have an eval file with at least twenty real cases, and a script that scores it.