Breaking into AI engineering: a roadmap from zero

A customer asks a company's help assistant whether they can still return a jacket bought five weeks ago. The assistant says yes: returns are accepted within 60 days. The real policy is 30 days. Nobody typed “60” into the assistant. A page from last winter's holiday promotion (“extended 60-day returns on gifts!”) had been added to the pile of pages the assistant searches, and it matched the customer's wording better than the real policy page did.

This category of mistake has already been tested in court. In 2024 a Canadian tribunal held Air Canada liable after its website chatbot told a passenger he could claim a bereavement discount after travelling, which the airline's actual policy did not allow (Moffatt v. Air Canada, 2024 BCCRT 149). The airline argued the chatbot was responsible for its own words. The tribunal did not accept that: the company owns what its assistant says.

Look at what a proper fix for the 60-day answer involves. Each step is a different part of the work, and together they are a fair description of what an AI engineer does all week.

• Retrieval. The search step, the “retrieval” half of retrieval-augmented generation, decides what the model gets to read before it answers. Making policy pages outrank marketing pages, or excluding marketing pages from policy questions altogether, is retrieval work.
• Data. A year-old promotion should never have been searchable. Someone has to decide which sources are allowed in, label each page with metadata such as “type: policy” or “expires: 15 January”, and remove pages when they go stale. This is the ingestion pipeline, and it is where most of the unglamorous hours go.
• Evaluation. The bug should have been caught before a customer saw it. A golden set of, say, 200 real customer questions, each with an answer the policy owner has signed off, gets re-run on every change. “Can I return a jacket after five weeks?” with the expected answer “No, the window is 30 days” would have failed loudly the day the promo page went in.
• Release. The fix itself can break other answers. Run the golden set, send the new version to 5% of customers first, watch how often people report a wrong answer, then go to 100%. That staged rollout is a canary release.

Notice what is missing: building or training a model. An AI engineer works on everything around a large language model that a lab trained: what it reads, what it is allowed to do, how you know it is right, and how changes reach people. The model is one component. The rest is careful engineering around a component that is right most of the time and confidently wrong the rest.

The handbook sorts readers into five levels. They describe what you can do, not a job title, and you move up by doing, not by asking permission. The estimates below assume 6–8 focused hours a week and, from Budding onwards, that you can already write basic code. If you are learning to program at the same time, roughly double the first three rows.

Curious to Budding is about a correct mental picture. A model reads text in small pieces called tokens, can only use what fits in its context window, and writes one piece at a time. Once those three ideas click, the 60-day bug stops being mysterious: the wrong page was in front of the model and the right one was not.

Budding to Mid is the biggest jump in usefulness and the step where most people stall. Building the demo takes a weekend. Knowing whether it works takes a golden set, an eval harness that scores every answer automatically, and the habit of reading the failures before changing anything. A Budding engineer says “it looks good”. A Mid engineer says “it gets 172 of 200 right, up from 151, and most of the remaining misses are pages that are out of date”.

Mid to Senior happens on the job, because the lessons only show up with real traffic: the bill at a million questions a month, the slowest 1% of answers, the provider retiring the model version you depend on. Senior to Staff is judgment that spans teams: choosing a simple fixed workflow over an agent when that is enough, or setting the evaluation standard five other teams reuse. The staff engineer page describes that role in detail.

Order matters more than speed. Each step gives you the vocabulary for the next, and the most common way to stall is to start at step five.

• 1. How a model answers (weeks 1–2). Tokens, the context window, next-token prediction and why models make things up. This handbook's Curious track covers it without code.
• 2. Prompting (week 3). Clear instructions, two or three worked examples, a fixed output format. It fixes more problems than beginners expect, and it costs nothing to change.
• 3. A first build (month 2). Take 20–50 documents you know well, such as a product manual or your team's wiki, and build a question-answering tool over them. Keep it small enough to check every answer yourself.
• 4. Retrieval and evaluation, together (months 3–5). Learn chunking and search alongside a 50–100 question golden set. Retrieval without measurement is guesswork; measurement without something to improve is homework.
• 5. Tools and agents (month 6 onwards). Letting the model call a search, a calculator or another system. Learn when a fixed sequence of steps beats an agent; for most business tasks, it does.
• 6. Production (on the job). Cost per answer, the slowest answers, staged rollouts and prompt injection, where text the model reads tries to give it orders.

In this order. All are free except the book, and all come from people who have built these systems for real users.

• Hamel Husain, “Your AI Product Needs Evals” (2024). Why measuring answers is the core of the job, drawn from consulting on shipped products. Read it right after your first build.
• Eugene Yan, “Patterns for Building LLM-based Systems & Products” (2023). Seven patterns, including evals, retrieval, guardrails and caching, each with production examples. The best single map of the field.
• Anthropic, “Building effective agents” (2024). Short and opinionated: start with simple, predictable workflows and add autonomy only when it measurably pays.
• Lilian Weng, “LLM Powered Autonomous Agents” (2023). The deeper explanation of planning, memory and tool use. Read it when agents reach the top of your list, not before.
• Chip Huyen, AI Engineering (O'Reilly, 2025). The book-length version of this roadmap, from foundations to deployment. Use it as a reference to dip into rather than a cover-to-cover read.

Then build something. Reading without a project fades within weeks; one small system you can explain end to end, including what it gets wrong, is worth more than any number of courses.