# DSPy tutorial: let code and a metric optimize your prompts

> DSPy is an open-source Python framework for programming a language model instead of hand-writing its prompt. You declare what goes in and out as a signature, pick a module such as ChainOfThought, write a metric that scores replies, and run an optimizer on a few examples. The optimizer then writes better instructions and picks examples for you.

Mindket editors · Published 2026-11-06 · Updated 2026-11-06 · https://mindket.com/learn/prompt-optimization/dspy-tutorial

## In short

- A signature declares what goes in and what comes out, and leaves the wording of the prompt to the optimizer.
- The docs say an optimizer can start from as few as five or ten training inputs, together with your program and your metric.
- Your metric is a plain Python function that returns a score, so write it and check it before any optimizer runs.
- The optimized program saves as plain JSON that you can open and read.

## Before you start

[The DSPy README](https://github.com/stanfordnlp/dspy/blob/main/README.md) calls it a framework for programming, rather than prompting, language models, and says the name stands for Declarative Self-improving Python. You describe the task in code, DSPy builds the prompt from it, and an optimizer can then tune that prompt against a score you define.

[Lessons 1 to 3](https://mindket.com/learn/prompt-optimization/test-prompts) wrote a prompt, improved it and tested the versions by hand. DSPy runs that loop for you: it tries many versions, scores each on your examples and keeps the best. You need Python, a model your API key can call and some examples. The package record on PyPI lists version 3.4.1, uploaded on 12 October 2026, for Python 3.10 up to but not including 3.16. Install it with `pip install dspy`.

This is the most technical lesson in the course. If you do not write Python, read it for the idea: your test set from lesson 3 becomes the training data, and your pass rule becomes a metric.

## The three stages, in this order

DSPy's overview names three stages and says to learn and apply them in this order. Its reason: it is unproductive to launch optimization runs on a poorly designed program or a bad metric.

**How DSPy says to build an AI system**

1. Programming. Define the task and its constraints, try a few examples and design a first pipeline. Start simple, perhaps with one ChainOfThought module.
2. Evaluation. Collect a development set, define your metric and score the system. That score is the baseline for everything after it.
3. Optimization. Use an optimizer to tune the prompts or the model weights against your metric.

Source: [DSPy docs on GitHub: Learning DSPy, an overview](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/index.md) (read 2026-11-06)

## Step 1: write a signature instead of a prompt

A signature is, in [DSPy's words](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/signatures.md), a declarative specification of the input and output behavior of a module. It tells the model what to do and leaves out how to ask. Field names carry meaning: a `question` is not an `answer`, and a `sql_query` is not `python_code`.

The shortest form is a string, such as `question -> answer` or `document -> summary`. When the task needs a rule, add instructions. Our suggestion is to put the rules of your own prompt there. For longer jobs, write a class with a docstring and use `desc` to describe a field.

The docs advise against hand-tuning the signature's keywords at the start. Optimizers will likely do it better, and the result carries over to other models.

````python
import dspy

# An inline signature, with instructions
toxicity = dspy.Predict(
    dspy.Signature(
        "comment -> toxic: bool",
        instructions="Mark as 'toxic' if the comment includes insults, harassment, or sarcastic derogatory remarks.",
    )
)
comment = "you are beautiful."
toxicity(comment=comment).toxic
# Output: False

# A class-based signature
from typing import Literal

class Emotion(dspy.Signature):
    """Classify emotion."""

    sentence: str = dspy.InputField()
    sentiment: Literal['sadness', 'joy', 'love', 'anger', 'fear', 'surprise'] = dspy.OutputField()
````

| Task | Signature written as a string |
| --- | --- |
| Question answering | question -> answer |
| Sentiment classification | sentence -> sentiment: bool |
| Summarization | document -> summary |
| Question answering over passages | context: list[str], question: str -> answer: str |
| Multiple choice with reasoning | question, choices: list[str] -> reasoning: str, selection: int |

## Step 2: choose a module and set the model

A module turns a signature into a model call and applies a technique to it. `dspy.Predict` is the basic one, and the [modules page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/modules.md) says every other module is built on it. Set the model once with `dspy.LM`. The docs' OpenAI example reads the key from `OPENAI_API_KEY`, and the [language models page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/language_models.md) shows `gemini/` and `anthropic/` prefixes with their own key variables.

Declare the module with a signature, call it with the input fields as keyword arguments and read the output field from the result. Swap `dspy.ChainOfThought` in for `dspy.Predict` and the model reasons step by step first. The docs say that in many cases this alone improves quality, and the result gains a `reasoning` field.

A bigger program is a class that holds several modules, with `__init__` to declare them and `forward` to call them. The programming page advises starting with a single module and adding steps only when what you see in the outputs calls for it.

````python
import dspy

# Authenticate via the OPENAI_API_KEY environment variable
lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

sentence = "it's a charming and often affecting journey."  # example from the SST-2 dataset.

# 1) Declare with a signature.
classify = dspy.Predict('sentence -> sentiment: bool')

# 2) Call with input argument(s).
response = classify(sentence=sentence)

# 3) Access the output.
print(response.sentiment)
# Output: True
````

| Module | What it does |
| --- | --- |
| dspy.Predict | The basic predictor. It does not change the signature. |
| dspy.ChainOfThought | Has the model think step by step before it answers. |
| dspy.ReAct | An agent that can use tools to fill the signature. |
| dspy.MultiChainComparison | Compares several ChainOfThought outputs to make a final prediction. |
| dspy.RLM | Explores a context too large for the prompt through a sandboxed Python REPL, with recursive sub-model calls. |
| dspy.majority | A function that votes and returns the most popular of several predictions. |

## Step 3: collect examples and write a metric

Your examples are `dspy.Example` objects, which behave like dictionaries. The [data page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/evaluation/data.md) shows `with_inputs`, which marks which fields are inputs, and the rest count as labels. Depending on the metric you need only inputs, or inputs and the final outputs. The docs say you almost never need labels for the steps in between.

A [metric](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/evaluation/metrics.md) is a function that takes an example and the program's output and returns a score. It also accepts an optional third argument, `trace`, which you can ignore at first. For short answers an exact match is enough. For long replies the docs suggest a small DSPy program that checks several properties, possibly with a model as the judge. They also say getting it right the first time is unlikely, so start simple and iterate.

Write the metric before you optimize anything, because it is hard to improve what you cannot define. Then score the unoptimized program on a development set with the `Evaluate` helper. It can run on several threads and prints a sample of inputs, outputs and scores. That score is your baseline.

````python
import dspy

article_summary = dspy.Example(article="This is an article.", summary="This is a summary.").with_inputs("article")

def validate_answer(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

from dspy.evaluate import Evaluate

# Set up the evaluator, which can be re-used in your code.
evaluator = Evaluate(devset=YOUR_DEVSET, num_threads=1, display_progress=True, display_table=5)

# Launch evaluation.
evaluator(YOUR_PROGRAM, metric=YOUR_METRIC)
````

- For most prompt optimizers the docs recommend an unusual split: 20% for training and 80% for validation, because prompt optimizers often overfit small training sets.
- GEPA is the exception. It follows the usual convention of the largest training set you can get and a validation set just big enough to reflect the real task.
- Keep a held-out test set apart from both, as the docs suggest once you move to optimization.

| Use | The docs say you can start with | The docs say to aim for |
| --- | --- | --- |
| Training inputs for an optimizer | 5 or 10 examples, inputs only | More, if you have it |
| A development set to score the program | 20 input examples | 200 |
| A training set for optimization | 30 examples | At least 300 |

## Step 4: pick an optimizer

An optimizer takes your program, your metric and some training inputs, and returns a better program. [The optimizers page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/optimizers.md) says it can tune the prompts, the model weights or both. Which one to start with depends on how much data you have, and the last column gives the docs' own advice.

GEPA is not in that advice. It is a reflective optimizer: a model reads how the program failed, using text feedback your metric returns along with the score, and proposes better instructions.

| Optimizer | What it tunes | The docs say to start with it when |
| --- | --- | --- |
| BootstrapFewShot | Few-shot examples, made by running your program and kept when the metric passes them | You have few examples, around ten |
| BootstrapFewShotWithRandomSearch | The same, tried several times with random search | You have 50 examples or more |
| MIPROv2 | Instructions and few-shot examples together, or instructions only | You want instructions only, or can run longer (40 trials or more) with 200 examples or more |
| GEPA | Instructions, through reflection on failures and text feedback | Not in the starting advice. Its metric returns a score and feedback text |
| BootstrapFinetune | The model's weights | A large model works and you need an efficient program, so you fine-tune a small one |

## Step 5: run MIPROv2 on the docs' example

The [MIPROv2 page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/api/optimizers/MIPROv2.md) has a complete example that optimizes a math program. It sets the model, builds the optimizer with `auto="medium"`, compiles a single `dspy.ChainOfThought("question -> answer")` on the GSM8K training set and saves the result. The `auto` setting takes `light`, `medium` or `heavy`.

MIPROv2 is the one to understand first, because it tunes the most. The page breaks its work into three steps.

````python
import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric

# Import the optimizer
from dspy.teleprompt import MIPROv2

# Initialize the LM
lm = dspy.LM('openai/gpt-4o-mini', api_key='YOUR_OPENAI_API_KEY')
dspy.configure(lm=lm)

# Initialize optimizer
teleprompter = MIPROv2(
    metric=gsm8k_metric,
    auto="medium", # Can choose between light, medium, and heavy optimization runs
)

# Optimize program
print(f"Optimizing program with MIPROv2...")
gsm8k = GSM8K()
optimized_program = teleprompter.compile(
    dspy.ChainOfThought("question -> answer"),
    trainset=gsm8k.train,
)

# Save optimize program for future use
optimized_program.save(f"optimized.json")
````

1. Bootstrap few-shot examples: run training examples through your program and keep the ones whose output is correct.
2. Propose instructions: a model writes candidates, given a summary of your data, a summary of your program's code, the bootstrapped examples and a random tip such as "be creative" or "be concise".
3. Search: Bayesian optimization decides which instructions and examples go together. It runs `num_trials` trials on minibatches and checks the best on the full validation set.

## Put it together for your own prompt

The script below joins the snippets above into one run for your own task. We put it together for this lesson from calls the docs show, and we have not run it. Swap in your own rules, examples and metric, and replace the model name with one your key can call.

The numbered steps after it are the order to work in. Keep the first run in `light` mode, which the docs use for their cheap example.

````python
import dspy
from dspy.evaluate import Evaluate

dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))

# 1. Your task as a program: your prompt's rules go in the instructions.
program = dspy.ChainOfThought(
    dspy.Signature(
        'question -> answer',
        instructions='PASTE THE RULES FROM YOUR PROMPT HERE',
    )
)

# 2. Your examples, with the input fields marked. Fill in your own rows.
trainset = [
    dspy.Example(question='...', answer='...').with_inputs('question'),
]
devset = [
    dspy.Example(question='...', answer='...').with_inputs('question'),
]

# 3. Your metric: the pass rule you wrote in lesson 3, as a function.
def validate_answer(example, pred, trace=None):
    return example.answer.lower() == pred.answer.lower()

# 4. Score the program before you change anything.
evaluator = Evaluate(devset=devset, num_threads=1, display_progress=True, display_table=5)
evaluator(program, metric=validate_answer)

# 5. Optimize in the cheapest mode, then score again on the same devset.
optimizer = dspy.MIPROv2(metric=validate_answer, auto='light')
optimized = optimizer.compile(program, trainset=trainset)
evaluator(optimized, metric=validate_answer)

# 6. Save it. The file is plain JSON, so you can read what the optimizer wrote.
optimized.save('optimized.json')
````

1. Write at least 50 examples of your task, with inputs and the output you expect. The docs say 20 examples can be useful as a development set and 30 can add value as a training set. Put 20 in the development list and 30 in the training list, and add more as you can.
2. Turn your prompt into a signature: the input and output fields, with your rules as the instructions.
3. Write the metric so it passes exactly what you would pass by hand in lesson 3.
4. Run the script. It scores the program, optimizes it with MIPROv2 in `light` mode, scores it again and saves it.
5. Open `optimized.json` and read the instructions and examples the optimizer chose. The docs say the file holds every parameter and step, so you can see what it generated.
6. Keep the change only if the development score rose and the failures you care about are gone.

## What the docs report, and what to expect

The optimizers page gives scores before and after for three runs and no run date. For the ReAct and Banking77 pairs it says an informal run similar to the example above it, on DSPy 2.5.29, an older release, raised the score. Those two examples use GPT-4o-mini, and the page names it outright for Banking77. For the RAG pair it points to a tutorial and gives no model, release or date. Read the bars as the rough size of a change, not a promise. Each pair is a different task and metric, so compare a bar only with the one next to it.

On time and cost, the docs call a simple optimization run about ten minutes and say the cost varies widely with the model, the dataset and the settings. They tell you to be careful with large models or large datasets.

**Scores before and after optimization, as the docs report them** Percent, as printed on the optimizers page, which gives no run date. The gold bars are the optimized programs. The ReAct and Banking77 pairs are informal runs on DSPy 2.5.29 with GPT-4o-mini examples, and the RAG pair comes from a tutorial that names no model or release. The last pair fine-tunes the model's weights, so it is not a prompt change.

- ReAct agent, before: 24%
- ReAct agent, after MIPROv2: **51%**
- RAG over StackExchange, before: 53%
- RAG over StackExchange, after: **61%**
- Banking77 classification, before: 66%
- Banking77 classification, after BootstrapFinetune: **87%**

Source: [DSPy docs on GitHub: Optimizers](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/optimizers.md) (read 2026-11-06)

## How the MIPROv2 announcement put it, and a talk from DSPy's list

The MIPROv2 page points to a thread on X as its visual explanation. It was posted on 21 June 2024 by Michael Ryan, who calls MIPROv2 "our new state-of-the-art optimizer for LM programs". His one-line summary is a good test of your understanding: if you can say what each of its three verbs does, you have the idea.

If you would rather watch before you code, DSPy's [Resources page](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/community/community-resources.md) lists talks, blog posts and podcasts about the project. One is Databricks' recording below.

> MIPROv2 proposes instructions, bootstraps demonstrations, and optimizes combinations.
>
> Michael Ryan (@michaelryan207), Calls MIPROv2 "our" optimizer, [X](https://x.com/michaelryan207/status/1804189184988713065) (read 2026-11-06)

[![Michael Ryan (@michaelryan207) on X, 21 June 2024: MIPROv2, our new state-of-the-art optimizer for LM programs, is live in DSPy. It is even faster, cheaper and more accurate than MIPRO. MIPROv2 proposes instructions, bootstraps demonstrations, and optimizes combinations.](https://mindket.com/blog/shots/x-michaelryan207-miprov2-thread.webp?v=2f923b0c6369)](https://x.com/michaelryan207/status/1804189184988713065)

Michael Ryan on X, 21 June 2024

[DSPy 3.0 and DSPy at Databricks](https://www.youtube.com/watch?v=grIuzesOwwU)

A recording on the Databricks channel, listed on DSPy's own Resources page as DSPy 3.0 and DSPy at Databricks.

> A screenshot of X's own embed of the post, taken on 6 November 2026, with the time in UTC. The post opens a thread, and its later posts are not in the screenshot. Select it to open the post on X.

## Common mistakes

- Launching an optimizer before the metric works. The docs call that unproductive, so check the metric on a few outputs you have already judged by hand.
- Scoring the optimized program on the examples it trained on. Keep a development or test set apart.
- Rewording the signature by hand before any run. Let the first optimizer run do that work.
- Trusting a metric that asks a model to judge, without reading some of its verdicts. The [judge lesson](https://mindket.com/learn/ai-evals/llm-as-a-judge) shows how to check one.
- Expecting the docs' numbers on your task. Two come from informal runs on DSPy 2.5.29, an older release, and the third from a tutorial, all on the docs' own models and data.
- Starting with a heavy run on a large model. Begin with `light`.

## Try it yourself

- Run `pip install dspy` and the docs' sentiment example with your own key, first with `dspy.Predict`, then with `dspy.ChainOfThought`.
- Turn the prompt you improved in lesson 2 into a signature with instructions.
- Write your first set of examples and a metric, and score the program before you change it.
- Run MIPROv2 in `light` mode, open the saved JSON and compare the two scores.

## Next course: Context engineering

You can now generate, improve, test and automatically tune a prompt. The [context engineering course](https://mindket.com/learn/context-engineering/what-is-context-engineering) widens the view, because a prompt is only one part of what an AI agent sees at each step.

## Check yourself

1. What does a DSPy signature declare?

   A. The exact words to send to the model
   B. What goes in and what comes out, not how to ask
   C. Which optimizer to run
   D. How much each call costs

   Answer: B. The docs call a signature a declarative specification of the input and output behavior of a module. It says what the model must do, not how to ask.

2. What three things does a DSPy optimizer take?

   A. A prompt, a model name and an API key
   B. A signature, a database and a budget
   C. A test file, a CSV and a judge
   D. A program, a metric and some training inputs

   Answer: D. The optimizers page says a typical optimizer takes your program, your metric and a few training inputs, which may be as few as 5 or 10.

3. What does optimized_program.save(path) write?

   A. A compiled binary
   B. A password-protected archive
   C. Plain-text JSON that you can read
   D. Only the final score

   Answer: C. The docs say the resulting file is plain-text JSON holding all the parameters and steps, so you can read what the optimizer generated.

4. What should be in place before you launch an optimization run?

   A. A well-designed program and a good metric
   B. A bigger model
   C. A held-out file of failures only
   D. The heavy setting

   Answer: A. The overview says it is unproductive to launch optimization runs on a poorly designed program or a bad metric.

## Frequently asked questions

### What is DSPy used for?

It is used to build programs out of language-model calls without hand-writing each prompt. Its README names simple classifiers, RAG pipelines and agent loops as examples, and in our reading the common thread is a task whose output you can score.

### How does DSPy optimize a prompt?

An optimizer runs your program on training examples, scores the results with your metric and keeps what scores best. Depending on the optimizer it writes new instructions, picks few-shot examples or both. MIPROv2 searches the combinations with Bayesian optimization.

### How many examples do I need for DSPy?

A handful is enough to start, and the inputs need no labels. The docs set larger targets for a development set and a training set.

### Is DSPy better than writing prompts by hand?

The project says it often is. Its signatures page says compiling led to better prompts than humans write in many cases, because optimizers can try more things and tune the metric directly, not because they are more creative. Check it on your own development set.

### Do I need to know Python to use DSPy?

Yes, DSPy is a Python library, so you write a short script. Most of the code in this lesson is a signature, a metric function and a few calls, copied from the DSPy docs.

### Does DSPy work with Claude and Gemini?

Yes. The docs' language models page shows OpenAI, Gemini and Anthropic models, plus Vertex AI, Azure OpenAI, Databricks and local models on a GPU server.

## Sources

- [DSPy on GitHub: README](https://github.com/stanfordnlp/dspy/blob/main/README.md) (read 2026-11-06)
- [DSPy docs on GitHub: Learning DSPy, an overview](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/index.md) (read 2026-11-06)
- [DSPy docs on GitHub: Programming in DSPy](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/overview.md) (read 2026-11-06)
- [DSPy docs on GitHub: Signatures](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/signatures.md) (read 2026-11-06)
- [DSPy docs on GitHub: Modules](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/modules.md) (read 2026-11-06)
- [DSPy docs on GitHub: Language models](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/programming/language_models.md) (read 2026-11-06)
- [DSPy docs on GitHub: Evaluation in DSPy](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/evaluation/overview.md) (read 2026-11-06)
- [DSPy docs on GitHub: Data](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/evaluation/data.md) (read 2026-11-06)
- [DSPy docs on GitHub: Metrics](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/evaluation/metrics.md) (read 2026-11-06)
- [DSPy docs on GitHub: Optimization in DSPy](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/overview.md) (read 2026-11-06)
- [DSPy docs on GitHub: Optimizers](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/learn/optimization/optimizers.md) (read 2026-11-06)
- [DSPy docs on GitHub: MIPROv2](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/api/optimizers/MIPROv2.md) (read 2026-11-06)
- [DSPy docs on GitHub: GEPA overview](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/api/optimizers/GEPA/overview.md) (read 2026-11-06)
- [DSPy docs on GitHub: Resources](https://github.com/stanfordnlp/dspy/blob/main/docs/docs/community/community-resources.md) (read 2026-11-06)
- [PyPI: dspy package record](https://pypi.org/pypi/dspy/json) (read 2026-11-06)
- [X: Michael Ryan on MIPROv2](https://x.com/michaelryan207/status/1804189184988713065) (read 2026-11-06)
- [YouTube: Databricks, DSPy 3.0 and DSPy at Databricks](https://www.youtube.com/watch?v=grIuzesOwwU) (read 2026-11-06)

[Browse GitHub projects](https://mindket.com/github)