Before you start
You need a spreadsheet and one task you care about. Nothing in this lesson needs a GPU or a line of code. The task should fit in a sentence, such as "turn a customer email into a ticket title of at most 12 words".
Lessons 2 to 4 build the data for that task, train a small open model on it and run the result, so the task you pick here carries through the whole course.
Sort each failure into context or behavior
OpenAI's accuracy guide puts every model problem on one of two axes. A context problem comes from a model that lacks knowledge because it was not in the training set, whose knowledge is out of date, or that needs your private information. A behavior problem shows up as inconsistent formatting, the wrong tone or style, or reasoning the model does not follow every time.
The guide explains the split with an exam. Fine-tuning is six months of classes, where many repeated examples teach how a concept works. Retrieval is the textbook on the desk, where you look the answer up. The two stack, and some jobs need both.
Sorting your own failures takes an afternoon and decides what you build next.
- Write down 20 inputs your current prompt handles badly. Take them from real use, not from imagination.
- Next to each one, write the output you wanted.
- Take the first failure, paste the missing document, rule or fact into the prompt and ask again. This paste test is the lesson's own shortcut for the guide's split.
- If the answer comes out right, mark the row "context". Retrieval fits that row, as in the RAG tutorial.
- If the facts are in the prompt and the answer still has the wrong format, tone or consistency, mark the row "behavior".
- Repeat for all 20 rows. A pile of context marks points to retrieval. A pile of behavior marks that survives adding examples to the prompt (the few-shot prompting lesson) is the case for fine-tuning.
What fine-tuning changes, and what it leaves alone
Supervised fine-tuning, the kind this course teaches, shows a model prompts paired with the answers you wanted and moves its weights toward them. OpenAI's fine-tuning guide lists where that works best: classification, nuanced translation, generating content in a specific format and correcting instruction-following failures.
The accuracy guide adds a second reason, efficiency. Training can reach the same accuracy with fewer tokens, or from a smaller model, because the instructions and examples no longer travel with every prompt.
| Your problem | Reach for first | What the sources say |
|---|---|---|
| Answers use the wrong format or tone, or ignore an instruction now and then | A prompt with examples, then fine-tuning | OpenAI: supervised fine-tuning is best for specific formats and instruction-following failures. |
| The model lacks your documents, or they change often | Retrieval (RAG) | Ovadia and colleagues, on three 7B models in 2023: retrieval consistently outperformed unsupervised fine-tuning, for old and new knowledge. |
| One narrow task runs thousands of times a day | Fine-tune a small model | OpenAI: fine-tuning can reach the same accuracy with fewer tokens or a smaller model. |
| You want the model to know new facts reliably | Retrieval, with fine-tuning only for the style around it | Gekhman and colleagues: new facts are learned slowly and then raise hallucinations. |
Why new facts are the wrong thing to train in
A 2024 study by Zorik Gekhman and colleagues fine-tuned PaLM 2-S on closed-book question answering and varied how many training examples held facts the model did not already know. Those examples were learned significantly slower than the ones that matched what the model knew. As they were eventually learned, they raised the model's tendency to hallucinate in a straight line. The authors read this as evidence that models gain most of their factual knowledge in pretraining, and that fine-tuning teaches them to use it more efficiently.
In 2023, Ovadia and colleagues compared unsupervised fine-tuning with retrieval on Llama2-7B, Mistral-7B and Orca2-7B. Fine-tuning helped a little. Retrieval did better on facts the model had met in training and on entirely new ones, and showing the model many rewordings of one fact during training eased the problem.
The same split explains why a small set of examples can still change behavior. The 2023 LIMA paper fine-tuned a 65B LLaMA model on 1,000 curated prompts and responses, with no reinforcement learning, and the model learned specific response formats from a handful of examples. In a human study its answers were equal to or preferred over GPT-4's in 43% of cases. The authors conclude that almost all knowledge in a language model is learned during pretraining.
1,000
curated prompts and responses used to fine-tune LIMA's 65B model
Source: Hugging Face Papers: LIMA: Less Is More for Alignment (read November 6, 2026)
43%
of cases where people rated LIMA's answer equal to or better than GPT-4's
Source: Hugging Face Papers: LIMA: Less Is More for Alignment (read November 6, 2026)
A task where fine-tuning won and retrieval hurt
OpenAI's accuracy guide works one example through all three tools: correcting Icelandic sentences, scored with BLEU, which rates how similar the output is to reference texts. GPT-4 with no examples scored 62. Three examples in the prompt lifted it to 70. Fine-tuning gpt-3.5-turbo on 1,000 examples scored 78, and fine-tuning GPT-4 on the same set scored 87. The guide names those two models and carries no date, so newer models may score differently.
Then the guide added retrieved examples from outside the training set to the fine-tuned GPT-4, and the score fell four points to 83. Its reading is that the task was a behavior problem, so the extra context added noise to a model that had already learned the task. If your pile from the sort above is mostly behavior, that is the pattern to watch for.
Source: OpenAI: Optimizing LLM Accuracy (read November 6, 2026)
What fine-tuning costs you
Training is not a one-off. The accuracy guide notes that with fine-tuning you rerun the process and manage your training and validation sets each time you tune again, and that this can bring regressions as an application grows.
The model can also forget. In a 2024 study, Dan Biderman and colleagues compared LoRA with full fine-tuning of Llama-2-7B on programming and math. In its standard low-rank settings, LoRA underperformed full fine-tuning on the target domain, and it kept the base model's skills on other tasks better. Lesson 3 uses LoRA, which trades some learning for less forgetting and less memory.
Hosted tuning can also go away. OpenAI's fine-tuning pages say the platform is closed to new users and that eligible existing customers can create training jobs until 6 January 2027, per the guide read on 6 November 2026. With an open model, the base weights and your adapter stay on your own disk.
Common mistakes
- Fine-tuning on your company documents to "teach the model" your facts.
- Starting to train before a prompt baseline and a set of scored examples exist.
- Training on tidy invented examples while the real inputs are messy. OpenAI calls non-representative training data one of the most common pitfalls.
- Adding more examples after the first 50 changed nothing. The fine-tuning guide advises rethinking the task or the prompt first.
- Judging the tuned model by a few chats instead of a held-out set.
Try it yourself
- Finish the 20-failure sheet and count the context rows against the behavior rows.
- For the behavior rows, write the answer you wanted for five of them. They become the first lines of lesson 2's dataset.
- Add three of them as examples in your prompt, then recount how many behavior failures remain.
- Write one sentence that starts "I will fine-tune because" and check that the ending names a behavior, not a fact.
Next
Lesson 2 turns the behavior rows into a training file: the chat format, how many examples to start with, a validation script and a held-out test set.
Check yourself
A model keeps answering in the wrong tone, although it has every fact it needs. Which lever fits first?
Show answer
C. Examples in the prompt, then fine-tuning if the tone still slips Tone and format are behavior problems, which OpenAI's guide puts on the axis that fine-tuning serves.
What did Gekhman and colleagues find about training examples that hold facts the model did not know?
Show answer
B. They were learned slower and, once learned, raised hallucinations Their abstract says new-knowledge examples are learned significantly slower and then linearly increase the tendency to hallucinate.
In OpenAI's Icelandic case study, what happened when retrieved examples were added to the fine-tuned GPT-4?
Show answer
D. The score fell to 83 The task was a behavior problem, so the extra context only added noise to a model that had learned the task.
What does the lesson's paste test check?
Show answer
A. Whether the answer turns right once the missing document is in the prompt, which marks a context problem A fixed answer means the model lacked context, so retrieval fits. A wrong format that stays wrong marks a behavior problem.
Frequently asked questions
Should I use RAG or fine-tuning?
Use RAG when the model lacks facts or the facts change, and fine-tuning when it has the facts and answers in the wrong way. OpenAI's guide calls the two additive, so a job can need both.
Can fine-tuning teach an LLM new facts?
Barely, and with side effects. Gekhman and colleagues found that examples with new facts were learned significantly slower, and once learned they raised hallucinations. Put changing facts in retrieval.
Is fine-tuning better than prompt engineering?
Not as a first step. OpenAI's guide says to squeeze as much accuracy as you can out of basic methods before reaching for retrieval or fine-tuning, and to start with a prompt and an eval set.
Can I use fine-tuning and RAG together?
Yes. If your app retrieves documents, OpenAI advises training on examples that include the retrieved text, so the model does not learn to use context from nothing.
How many examples does fine-tuning need?
OpenAI's fine-tuning guide sets a minimum of 10 and sees improvements from 50 to 100 examples. It advises starting small and evaluating before adding more. The right number depends on the task.
Why fine-tune an open model instead of a hosted one?
You keep the result. A LoRA adapter is a small file you load on top of the base model, and the weights stay on your disk, whatever happens to a vendor's platform.
Sources
- OpenAI: Optimizing LLM Accuracy https://developers.openai.com/api/docs/guides/optimizing-llm-accuracy
- OpenAI: Supervised fine-tuning https://developers.openai.com/api/docs/guides/supervised-fine-tuning
- OpenAI: Model optimization https://developers.openai.com/api/docs/guides/model-optimization
- Hugging Face Papers: Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? https://huggingface.co/papers/2405.05904
- Hugging Face Papers: Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs https://huggingface.co/papers/2312.05934
- Hugging Face Papers: LIMA: Less Is More for Alignment https://huggingface.co/papers/2305.11206
- Hugging Face Papers: LoRA Learns Less and Forgets Less https://huggingface.co/papers/2405.09673
- Wikipedia: BLEU https://en.wikipedia.org/wiki/BLEU