Mid track
Making it good
You've shipped something with an LLM. Now make retrieval, agents and evaluation actually reliable.
Mid · Building real systems
0 of 12 done · about 9 h totalStart: Prompt, retrieve or fine-tune? A decision guide
- Read · 7 minPrompt, retrieve or fine-tune? A decision guideDiagnose the failure first, then pick the technique. A worked claims-triage case shows where fine-tuning pays in 2026: cost, not knowledge.
- Work through a section · ~2 hChunking & EmbeddingsThe least glamorous section and the one that decides your ceiling.
- Read · 6 minHybrid search and reranking: a field guideDense and sparse retrieval fail in opposite ways. How Reciprocal Rank Fusion combines them in 15 lines, where ColBERT and SPLADE fit, and what a Cross-encoder costs per 100 candidates on CPU and GPU.
- Learn a term · quiz · ~8 minBM25The default keyword ranking formula in Lucene, Elasticsearch, OpenSearch and Vespa: rewards rare terms, saturates repeats, normalises length.
- Learn a term · quiz · ~8 minHybrid searchRun keyword (BM25) and vector search in parallel and merge the lists, usually with RRF, so each covers the other's misses.
- Learn a term · quiz · ~8 minCross-encoderReads the query and one document together and outputs a relevance score: far more precise than vector similarity, too slow to run over a whole corpus.
- Work through a section · ~2 hRetrieval & RankingDense vs sparse, hybrid, rerankers, top-k and the vocabulary interviewers check for.
- Learn a term · quiz · ~8 minMCP (Model Context Protocol)Model Context Protocol: an open JSON-RPC standard connecting AI apps (hosts) to servers exposing tools, resources and prompts. Linux Foundation since Dec 2025.
- Work through a section · ~2 hAgent fundamentalsWhat makes something an agent, tool use, the agent loop, and when a workflow is better.
- Read · 9 minLLM-as-judge without fooling yourselfA model can grade open-ended answers at scale, but only after you have measured its true-positive and true-negative rates against people and corrected the pass rate it reports.
- Learn a term · quiz · ~8 minLLM-as-judgeUsing an LLM with a rubric to grade outputs (faithfulness, relevance, correctness) so evaluation scales beyond human review.
- Work through a section · ~2 hEvaluationGolden sets, rubrics, LLM-as-judge, human review and the statistics behind a real improvement.