Instruction tuning
Supervised fine-tuning on (instruction, good response) pairs that turns a base model that continues text into an assistant that follows requests.
A freshly pretrained model only continues text. Ask it 'What is the capital of France?' and it might reply with more quiz questions. Instruction tuning fixes that by training it on thousands of examples of instructions paired with good answers.
Afterwards the model understands that a question wants an answer and a request wants the task done. It is a form of Fine-tuning and usually comes before RLHF (reinforcement learning from human feedback) or Preference optimization (DPO).
A well-read graduate on their first day at a job: they know plenty, but someone has to show them, with examples, how the team expects questions to be answered.
Using instruction tuning to inject facts. SFT on a few thousand examples is poor at adding reliable knowledge and can increase confident hallucination on topics just beyond the data; use Retrieval-Augmented Generation (RAG) for facts.
Instruction tuning, or supervised fine-tuning (SFT), uses the same next-token loss as pretraining on curated (prompt, response) pairs, with prompt tokens masked out of the loss so the model learns to produce responses, not prompts. Data is rendered through a chat template — role tokens for system, user, assistant and tool turns — and the exact template must match at Inference.
Quality and diversity dominate quantity: LIMA reached strong assistant behaviour with 1,000 curated examples on a 65B base; FLAN showed that instruction-tuning across many tasks improves zero-shot performance on unseen ones. Most SFT data is now synthetic (generated by a stronger model, then filtered). SFT imitates its demonstrations, errors included; preference methods then teach which of several plausible answers is better. Product teams apply it as a LoRA & parameter-efficient fine-tuning on an instruct checkpoint: typical settings are lr 1e-4–2e-4 for LoRA (1e-5–2e-5 for full fine-tuning), 2–3 epochs, sequence packing, and a held-out set split by source.
LoRA (rank 16, alpha 32, all linear layers) on the vendor's instruct checkpoint; loss on assistant tokens only; the tokenizer's own apply_chat_template for both training and serving; lr 1e-4, 2–3 epochs, cosine decay; 10% held out by source document; compare against a few-shot prompt baseline before shipping.
Developer-docs assistant: the base instruct model answers API questions in prose, but the team needs a fixed format (one-line answer, code block, link to the doc section). SFT with LoRA rank 16 on 3,000 examples written by docs engineers (~2.4M tokens × 3 epochs) runs in ~1 hour on one H100. Format compliance on 400 held-out questions goes from 61% to 97%; factual accuracy is unchanged, because SFT taught shape, not knowledge.
Asked 'walk me through how a base model becomes a chat model', name the stages and what each buys: pretraining (knowledge, fluency) → SFT (format, following requests) → preference tuning (which answer is better) → RL with verifiable rewards (reasoning). Then say when you would do SFT yourself: when prompting cannot hold a format or behaviour at your volume. Expect: 'how many examples?' — 1k–10k clean ones; measure the curve at 1k, 3k, 10k.
Base model
- Continues any text
- May ignore the question
- No chat roles
- Knows facts, not manners
After SFT
- Answers the request
- Holds the format
- Uses chat template
- Ready for preference tuning
Numbers worth knowing
- LIMA
- 1,000 examplesenough for strong format/alignment on a 65B base
- Typical product SFT set
- 1k–10k examplesLoRA on an instruct model
- LoRA SFT learning rate
- 1e-4 to 2e-4full fine-tune uses 1e-5 to 2e-5
How to decide
| When | Do this |
|---|---|
| A few-shot prompt already holds the format at ≥ 95% | Do not fine-tune; ship the prompt with Prompt caching. |
| Format or tone must hold at millions of calls and prompts are long | SFT a smaller model with LoRA to drop the examples from every call. |
| Behaviour is right but answers are wrong on facts | Fix retrieval; SFT will not add reliable knowledge. |
Cost maths
Training FLOPs ≈ 6 × params × tokens × epochs (LoRA saves memory, not much compute). 8B × 2.4M tokens × 3 epochs ≈ 3.5e17 FLOPs ≈ 15 minutes of H100 at 40% utilisation; with data loading and evals, budget 1 GPU-hour ≈ $3 (illustrative). The 3,000 expert-written examples at 10 minutes each and $80/hour cost $40,000. Data is ~99% of the cost.
Operating it
- Few-shot prompt exceeds ~2k tokens on a high-volume route → fine-tune to remove it
- Format compliance < 95% with the best prompt
- Held-out loss rising after epoch 1 → fewer epochs or more diverse data
- Role-marker or template-token leakage in outputs > 0.1%
- Format compliance on a daily sample < 95%
- Golden-set accuracy delta vs base instruct model < 0 on any category (regression)
When it goes wrong
After deploying a fine-tuned support model, 12% of replies contained stray role markers and continued writing the user's next message.
An output filter counting role-token strings spiked on deploy day; the eval harness, which used a different code path, had passed.
Training data was rendered with a ChatML-style template while the serving stack applied the base model's native template, so the model never saw its learned end-of-turn token.
Used the tokenizer's apply_chat_template in both training and serving, added a test asserting byte-identical rendering, and ran the eval harness through the production serving path.
Related terms
Where it shows up
Sources
- Training language models to follow instructions with human feedback — Ouyang et al., 2022paper
- Finetuned Language Models Are Zero-Shot Learners — Wei et al., 2021paper
- LIMA: Less Is More for Alignment — Zhou et al., 2023paper
- Chat templates — Hugging Face, 2026docs
Check yourself
Three questions. Every option is explained once you answer.