Budding track
Your first real builds
You can code and want working mental models plus three small builds: a retrieval pipeline, a tool-using agent and an eval set.
Budding · Starting out
0 of 14 done · about 5.7 h totalStart: The three dials: tokens, temperature and context
- Read · 7 minThe three dials: tokens, temperature and contextTwo bugs show up in nearly every first LLM integration. Both trace back to the same three settings, and both are fixable in a few lines of code.
- Learn a term · quiz · ~8 minContext windowThe maximum number of tokens, prompt plus output, a model can handle in one call. Anything outside it does not exist for the model.
- Learn a term · quiz · ~8 minTemperature & samplingA sampling setting that sharpens (low) or flattens (high) next-token probabilities, trading consistency for variety; it does not change what the model knows.
- Learn a term · quiz · ~8 minPrompt & system promptThe full input sent to an LLM: instructions, examples, retrieved context and the user's message. The main lever for controlling behaviour.
- Work through a section · ~2 hHow LLMs workTokens, transformers, prompting and why models make things up.
- Read · 12 minYour first RAG pipeline in pure PythonChunk, vectorise, rank and build a cited prompt in ~150 lines of standard-library Python, test it with pytest, then swap in a real Embedding model and LLM without touching the rest.
- Learn a term · quiz · ~8 minChunkingSplitting documents into passages small enough to embed and retrieve. Default: ~200–400 tokens on structural boundaries, no overlap.
- Learn a term · quiz · ~8 minCosine similarityA score from −1 to 1 measuring the angle between two vectors, ignoring their length: the standard way to compare embeddings.
- Work through a section · ~2 hRAG FundamentalsWhy RAG exists, when not to build it, and what happens inside a query.
- Read · 11 minBuild your first tool-using agent (offline, in pure Python)A minimal agent loop that speaks the real Messages API shape: tool calls linked to results by id, several calls per turn, argument validation that does not swallow your bugs, a step budget, pytest, and the same loop against Claude.
- Learn a term · quiz · ~8 minTool use & function callingThe model emits a structured call (tool name + JSON arguments); your application validates and runs it, then returns the result to the model.
- Learn a term · quiz · ~8 minThe agent loop (ReAct)The core cycle of every agent: model decides, host runs a tool, result is appended, repeat until a final answer or a stop condition.
- Read · 10 minYour first eval set in an afternoonThirty to fifty real examples, five check types, a Wilson interval on the score, and a comparison that respects noise — in plain Python with no frameworks.
- Learn a term · quiz · ~8 minRecall@kOf all the relevant documents for a query, the fraction that appear in the top k results. The ceiling on what a RAG system can answer.