# Fine-tuning dataset: build, check and split your training data

> A fine-tuning dataset is a file of example conversations, one per line, where the last message is the answer you want the model to give. Start with about 50 realistic examples, check the file with a script, and hold some back as a test set. Quality counts for more than quantity: the LIMA paper fine-tuned on only 1,000 curated examples.

Mindket editors · Published 2026-11-06 · Updated 2026-11-06 · https://mindket.com/learn/fine-tuning/fine-tuning-dataset

## In short

- Write the first batch from real inputs, each paired with the answer you would ship.
- Run a format check and a token count before the first training step.
- Split off the test file first and leave it alone until the final score.
- Match the prompt shape in training to the prompt your application sends.

## Before you start

You need Python, the `datasets` library (`pip install datasets`) and the behavior rows from lesson 1. The steps below need a real batch of examples. Start writing them now. Whether the task can be learned shows only once they exist.

## One example per line

Fine-tuning data is a JSONL file: one complete JSON object on every line, with no commas between lines and no outer list. Nothing else goes in the file. OpenAI's [fine-tuning guide](https://developers.openai.com/api/docs/guides/supervised-fine-tuning) asks for the chat completions format, a `messages` list of objects that each have a `role` and a `content`. TRL, the library lesson 3 trains with, reads the same layout as a conversational dataset.

Line one comes from OpenAI's [data preparation cookbook](https://developers.openai.com/cookbook/examples/chat_finetuning_data_prep). Line two is this lesson's own, for the ticket-title task from lesson 1. `load_dataset("json", data_files="all.jsonl")` reads such a file in Hugging Face Datasets, whose [docs](https://huggingface.co/docs/datasets/loading) call JSON Lines the most efficient JSON layout.

````json
{"messages": [{"role": "system", "content": "You are a happy assistant that puts a positive spin on everything."}, {"role": "user", "content": "I fell off my bike today."}, {"role": "assistant", "content": "It's great that you're getting exercise outdoors!"}]}
{"messages": [{"role": "system", "content": "Write a ticket title of at most 12 words."}, {"role": "user", "content": "Hi, the invoice PDF in my account shows last year's logo and I need the new one for my accountant."}, {"role": "assistant", "content": "Invoice PDF shows the old logo instead of the new one"}]}
````

1. Save one example per line in a file called `all.jsonl`. Each line is a `messages` list that ends with the assistant message you want.
2. Reuse the system and user wording your application sends in production. The [accuracy guide](https://developers.openai.com/api/docs/guides/optimizing-llm-accuracy) calls training data that differs subtly in formatting from production inputs one of the most common pitfalls.
3. Write the first batch yourself, or take it from logs and expert answers if you have them. OpenAI lists historical, expert and logged data as good sources, and asks for questions and answers that are specific and clear.

## How TRL reads the file

TRL's [SFT trainer](https://huggingface.co/docs/trl/sft_trainer) accepts two dataset types, language modeling and prompt-completion, each in a standard (plain text) or a conversational form. Given a conversational dataset, it applies the model's chat template for you. Which tokens count toward the loss depends on the type.

Set `assistant_only_loss=True` when the system and user turns are boilerplate you do not want the model to copy. It needs a chat template with `{% generation %}` markers. TRL patches the template for known families such as Qwen3, and the docs say to check any other model's.

| Layout | One line looks like | Loss counted on |
| --- | --- | --- |
| Conversational language modeling | `{"messages": [{"role": "user", ...}, {"role": "assistant", ...}]}` | The whole sequence by default, or the assistant messages only with `assistant_only_loss=True` |
| Conversational prompt-completion | `{"prompt": [{"role": "user", ...}], "completion": [{"role": "assistant", ...}]}` | The completion only, by default |

## How many examples to start with

OpenAI's guide sets the floor at 10 examples and reports improvements from 50 to 100. Its advice is to begin at the low end of that range with well-crafted demonstrations, evaluate, and add more only if the score rises. A first set that changes nothing is a signal to rethink the task or the prompt. Add training data only after that.

The authors of [LIMA](https://huggingface.co/papers/2305.11206) went the other way in 2023 and curated 1,000 examples for a 65B LLaMA model, which shows that a small, careful set can teach a format. Your own count comes from your first evaluation, not from either of these numbers, so write the test file before you decide how many more examples to add.

**Example counts named in the sources** Training examples, from OpenAI's fine-tuning guide and the LIMA paper. The gold bar is where OpenAI suggests starting.

- OpenAI: the minimum: 10 examples
- OpenAI: where to start: **50 examples**
- OpenAI: top of the range where gains show: 100 examples
- LIMA: its curated set: 1,000 examples

Sources: [OpenAI: Supervised fine-tuning](https://developers.openai.com/api/docs/guides/supervised-fine-tuning) (read 2026-11-06), [Hugging Face Papers: LIMA: Less Is More for Alignment](https://huggingface.co/papers/2305.11206) (read 2026-11-06)

## Check the file before you train

A format error costs seconds to find now and an hour to find after training starts. OpenAI's cookbook is marked archived and was written for the chat format of its day, yet its checks still cover the basics of a chat file. The loop below counts seven kinds of error: a non-object entry, a missing `messages` list, a message without `role` or `content`, and an unrecognized key. It also flags an unrecognized role, empty content, and an example with no assistant message.

A clean file prints "No errors found". Otherwise, fix each error type it names and run it again.

````python
import json
from collections import defaultdict

with open("all.jsonl", "r", encoding="utf-8") as f:
    dataset = [json.loads(line) for line in f]

# Format error checks
format_errors = defaultdict(int)
for ex in dataset:
    if not isinstance(ex, dict):
        format_errors["data_type"] += 1
        continue
    messages = ex.get("messages", None)
    if not messages:
        format_errors["missing_messages_list"] += 1
        continue
    for message in messages:
        if "role" not in message or "content" not in message:
            format_errors["message_missing_key"] += 1
        if any(k not in ("role", "content", "name", "function_call", "weight") for k in message):
            format_errors["message_unrecognized_key"] += 1
        if message.get("role", None) not in ("system", "user", "assistant", "function"):
            format_errors["unrecognized_role"] += 1
        content = message.get("content", None)
        function_call = message.get("function_call", None)
        if (not content and not function_call) or not isinstance(content, str):
            format_errors["missing_content"] += 1
    if not any(message.get("role", None) == "assistant" for message in messages):
        format_errors["example_missing_assistant_message"] += 1
if format_errors:
    print("Found errors:")
    for k, v in format_errors.items():
        print(f"{k}: {v}")
else:
    print("No errors found")
````

## Count tokens against max_length

A model reads tokens, and the trainer cuts every example at `max_length`. In TRL's `SFTConfig` that is 1024 by default, and truncation keeps the start of the sequence, so the cut falls on the end, where the answer sits. TRL's [memory page](https://huggingface.co/docs/trl/reducing_memory_usage) adds that a value too small discards a significant share of your tokens, and one too large can spike memory.

The snippet below is this lesson's own. It turns each example into text with the model's chat template, counts the tokens and tells you how many examples go over the limit.

````python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
limit = 1024  # SFTConfig's default max_length

too_long = 0
for ex in dataset:
    text = tokenizer.apply_chat_template(ex["messages"], tokenize=False)
    if len(tokenizer(text, add_special_tokens=False)["input_ids"]) > limit:
        too_long += 1
print(too_long, "of", len(dataset), "examples are over", limit, "tokens")
````

## Hold out a test set first

OpenAI's accuracy guide recommends a hold-out set to catch overfitting, and its fine-tuning guide says to carve it out of the data you collected before you run a job. The script below, also this lesson's own, drops duplicate prompts first so one input cannot land on both sides, then splits.

`train_test_split` in [Hugging Face Datasets](https://huggingface.co/docs/datasets/process) shuffles by default and returns a train and a test split. A tenth of a small file leaves only a handful of test rows, so the script holds back a fifth. That fraction is the lesson's own pick.

Leave `test.jsonl` alone while you tune. Lesson 4 scores the base model and the tuned model on it.

````python
import json
from datasets import Dataset

seen, rows = set(), []
for ex in dataset:
    key = json.dumps(ex["messages"][:-1], sort_keys=True)  # everything except the answer
    if key not in seen:
        seen.add(key)
        rows.append(ex)

splits = Dataset.from_list(rows).train_test_split(test_size=0.2, seed=42)
splits["train"].to_json("train.jsonl")
splits["test"].to_json("test.jsonl")
````

## Common mistakes

- Training on tidy sentences when users send messy emails.
- Writing answers that need facts the base model lacks. [Gekhman and colleagues](https://huggingface.co/papers/2405.05904) found in 2024, with PaLM 2-S, that such examples are learned slowly and raise hallucinations.
- Leaving the longest examples above `max_length`, so the trainer cuts off their answers.
- Paraphrasing examples to grow the file before you split it, which puts near copies of test rows into training.
- Tuning the dataset against the test file until it stops being a test.

## Try it yourself

- Write a handful of examples in `all.jsonl` and run the format check until it prints "No errors found".
- Add the length check and note the longest example.
- Grow the file to the starting size above, run the split script and read the rows in `test.jsonl`.
- Write down the score you would accept on `test.jsonl` before any training.

## Next

[Lesson 3](https://mindket.com/learn/fine-tuning/lora-fine-tuning) trains a LoRA adapter on `train.jsonl` with TRL, and shows what each setting costs in GPU memory.

## Check yourself

1. Which layout is a valid JSONL training file?

   A. One JSON object per line, each with a messages list
   B. One JSON array that holds every example
   C. A CSV with prompt and answer columns
   D. A Markdown file with a heading per example

   Answer: A. OpenAI and TRL both read one complete JSON object on every line, with a messages list in the chat format.

2. Where does OpenAI's fine-tuning guide suggest starting on the number of examples?

   A. 10, the minimum
   B. 1,000, as in LIMA
   C. 50 well-crafted demonstrations, then evaluate
   D. 10,000, to be safe

   Answer: C. The guide recommends 50 well-crafted examples and adding more only if the results improve.

3. An example is longer than SFTConfig's max_length of 1024. What does TRL do by default?

   A. It drops the example from the dataset
   B. It stops with an error
   C. It splits the example into two
   D. It cuts the end of the example, which can remove the answer

   Answer: D. Truncation keeps the start of the sequence, so the end, where the answer sits, is what gets cut.

4. Why split off the test file before you train?

   A. It makes training run faster
   B. The score is then measured on examples the model never saw
   C. TRL refuses to start without a validation split
   D. It removes duplicate examples

   Answer: B. OpenAI advises a hold-out set to detect overfitting, which only works on rows the model was not trained on.

## Frequently asked questions

### How many examples do I need to fine-tune an LLM?

Enough to move your test score. OpenAI's guide says to start with a small set of well-crafted examples and evaluate it. If that set changes nothing, rethink the task before writing more.

### What format does fine-tuning data need?

For chat models, JSONL with one `messages` list per line, each message holding a `role` and `content`. TRL also accepts prompt and completion pairs, and plain `text` for language modeling.

### Can I use a CSV file instead of JSONL?

Datasets reads CSV too, and a CSV with prompt and completion columns matches TRL's standard prompt-completion format. OpenAI's fine-tuning guide asks for JSONL, and nested message lists fit it better.

### Should every example include a system prompt?

Only if production sends one. Then word it the same way. Examples that differ from what the model sees in production are a common pitfall, per OpenAI.

### How big should the test set be?

The sources name no size. OpenAI advises a hold-out set to detect overfitting. This lesson holds back a fifth while the dataset is small, so the score rests on more than a handful of rows.

### Can I train on my chat logs?

Yes, once you clean them. OpenAI lists logged data among good sources and describes prompt baking: log prompt inputs and outputs during a pilot, then prune them into a training set. Remove personal data and bad replies first.

## Sources

- [OpenAI: Supervised fine-tuning](https://developers.openai.com/api/docs/guides/supervised-fine-tuning) (read 2026-11-06)
- [OpenAI: Optimizing LLM Accuracy](https://developers.openai.com/api/docs/guides/optimizing-llm-accuracy) (read 2026-11-06)
- [OpenAI Cookbook: Data preparation and analysis for chat model fine-tuning](https://developers.openai.com/cookbook/examples/chat_finetuning_data_prep) (read 2026-11-06)
- [Hugging Face TRL: Dataset formats and types](https://huggingface.co/docs/trl/dataset_formats) (read 2026-11-06)
- [Hugging Face TRL: SFT Trainer](https://huggingface.co/docs/trl/sft_trainer) (read 2026-11-06)
- [Hugging Face TRL: Reducing memory usage](https://huggingface.co/docs/trl/reducing_memory_usage) (read 2026-11-06)
- [Hugging Face Datasets: Load](https://huggingface.co/docs/datasets/loading) (read 2026-11-06)
- [Hugging Face Datasets: Process](https://huggingface.co/docs/datasets/process) (read 2026-11-06)
- [Hugging Face Papers: LIMA: Less Is More for Alignment](https://huggingface.co/papers/2305.11206) (read 2026-11-06)
- [Hugging Face Papers: Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?](https://huggingface.co/papers/2405.05904) (read 2026-11-06)

[Browse MCP servers](https://mindket.com/mcp-servers)