# Chain of thought prompting: when to ask for steps and when to set the effort

> Chain of thought prompting asks a model to write out its reasoning before the final answer, through worked examples or a line such as "Let's think step by step". It helps models that answer straight away. Reasoning models already think before they reply, so OpenAI advises against these prompts there. On those, you set the reasoning effort instead and judge the final answer.

Mindket editors · Published 2026-11-06 · Updated 2026-11-06 · https://mindket.com/learn/prompt-engineering-advanced/chain-of-thought-prompting

## In short

- Ask a fast model for its reasoning in a tagged block, and strip that block before anyone reads the reply.
- Leave step-by-step instructions out of prompts for reasoning models, and tune the effort once the goal is clear.
- On both APIs, budget output tokens for thinking you never see.
- Reach for a think tool only in long tool chains, and only after extended thinking falls short.
- Treat a visible chain as a clue to check, never as proof the answer holds.

## Before you start

A chain of thought is the reasoning a model writes before its answer. The 2022 paper that named it, [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://huggingface.co/papers/2201.11903), defined it as a series of intermediate reasoning steps.

Reasoning models produce that chain without being asked. OpenAI's reasoning models, such as GPT-5.5, and Claude with extended thinking both reason before they reply, and [choosing an AI model](https://mindket.com/learn/ai-basics/choose-an-ai-model) explains how they differ from fast models. You need an API key for the exercises and the code.

## Ask a fast model to think out loud

Chapter 6 of Anthropic's [interactive prompting tutorial](https://github.com/anthropics/prompt-eng-interactive-tutorial/blob/master/Anthropic%201P/06_Precognition_Thinking_Step_by_Step.ipynb) asks Claude to name a famous movie starring an actor born in 1956. The tutorial uses it as a case where the direct answer comes out wrong. Its fix is one extra sentence: brainstorm some actors and their birth years in a `brainstorm` tag, then answer.

The tutorial adds a rule worth keeping. Thinking only counts when it's out loud. A prompt that asks the model to think but print only the answer gets no thinking at all, because the reasoning is the text the model writes.

````text
# Prompt 1
Name a famous movie starring an actor who was born in the year 1956.

# Prompt 2
Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.
````

1. Pick a model that answers without reasoning first, such as GPT-5.5 with the reasoning effort set to `none`.
2. Send prompt 1 and write down the film, the actor and the birth year the reply gives.
3. Look up that birth year yourself.
4. Send prompt 2 in a fresh conversation. The reply should list actors and years inside the tag before it names a film.
5. In your code, keep only the text after the closing tag, so people see the answer and not the brainstorm.

## Why spelled-out steps helped models in 2022

The original paper put the reasoning into few-shot examples, the technique from [the previous lesson](https://mindket.com/learn/prompt-engineering-advanced/few-shot-prompting). Each example showed a question, the worked steps and the answer. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a benchmark of math word problems, ahead of a fine-tuned GPT-3 with a verifier.

Later in 2022, [Large Language Models are Zero-Shot Reasoners](https://huggingface.co/papers/2205.11916) dropped the examples. Adding "Let's think step by step" before each answer lifted InstructGPT's text-davinci-002 on the same kind of problems, as the chart shows.

Both papers measured models that answered straight away unless prompted to reason, so the chart shows what one sentence did for such a model in 2022. It predicts nothing about today's models. [The basics lesson](https://mindket.com/learn/prompt-engineering/prompt-engineering-basics) explains why current models rarely need the line.

**Accuracy of text-davinci-002, plain zero-shot vs "Let's think step by step"** From the 2022 paper by Kojima and others. The gold bars add the step-by-step line before the answer.

- MultiArith, plain: 17.7%
- MultiArith, step by step: **78.7%**
- GSM8K, plain: 10.4%
- GSM8K, step by step: **40.7%**

Source: [Hugging Face Papers: Large Language Models are Zero-Shot Reasoners (2022)](https://huggingface.co/papers/2205.11916) (read 2026-11-06)

## What changes on a reasoning model

OpenAI's [reasoning guide](https://developers.openai.com/api/docs/guides/reasoning) describes reasoning tokens, which the model uses to break down the prompt and weigh approaches before the visible answer. The API never returns them. They still take room in the context window and are billed as output tokens.

Since the model reasons anyway, the [reasoning best practices](https://developers.openai.com/api/docs/guides/reasoning-best-practices) say to avoid chain-of-thought prompts. Telling it to think step by step or explain its reasoning is unnecessary, and the guide warns it may not help and can sometimes hinder. What OpenAI wants instead is a clear goal, strong constraints and an explicit output contract, without every intermediate step written out.

How much the model thinks is a setting on the request. OpenAI calls it the reasoning effort. The guide's example asks for a small bash script at low effort.

````python
from openai import OpenAI

client = OpenAI()

prompt = """
Write a bash script that takes a matrix represented as a string with
format '[1,2],[3,4],[5,6]' and prints the transpose in the same format.
"""

response = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "low"},
    input=[{"role": "user", "content": prompt}],
)

print(response.output_text)
````

| Setting | OpenAI (Responses API) | Claude (Messages API) |
| --- | --- | --- |
| Effort parameter | `reasoning.effort` | `output_config.effort` |
| Values | none, minimal, low, medium, high, xhigh, max, depending on the model | low, medium, high, xhigh, max |
| Default | medium on GPT-5.5, GPT-5.6 and GPT-6.1 Sol | high |
| Seeing the reasoning | A summary, if you set `reasoning.summary` to `auto` | A summary, if you set `display` to `summarized` |
| Token room | Reserve at least 25,000 tokens for reasoning and output at first | Thinking counts toward `max_tokens` |

## Set the effort on Claude

Anthropic's [extended thinking cookbook](https://github.com/anthropics/claude-cookbooks/blob/main/extended_thinking/extended_thinking.ipynb), written for Claude Sonnet 5.5, sends a scheduling puzzle: five talks, five slots and five constraints. Thinking there is adaptive. Claude decides per request how much to think, and the effort setting steers it. The older form with a fixed `budget_tokens` returns a 400 error on this model.

The call below is the cookbook's. Only the print loop is this lesson's own, standing in for the cookbook's helper function.

````python
import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=16000,  # thinking tokens count toward max_tokens, so leave headroom
    thinking={
        "type": "adaptive",
        "display": "summarized",  # default is "omitted" (empty thinking text)
    },
    output_config={"effort": "xhigh"},  # low | medium | high (default) | xhigh | max
    messages=[
        {
            "role": "user",
            "content": "Five talks (A, B, C, D, E) must be scheduled in five consecutive slots. Constraints: A comes before C; B is not adjacent to D; E is neither first nor last; C is immediately before or after E; D comes before A. List every valid ordering and explain how you know there are no others.",
        }
    ],
)

for block in response.content:
    if block.type == "thinking":
        print("THINKING (summary):", block.thinking)
    elif block.type == "text":
        print("ANSWER:", block.text)
````

- The thinking text is a summary, yet the bill covers the full reasoning. Leave `display` out and the blocks come back empty, still billed.
- Easy prompts and low effort can return no thinking block at all, so never assume the first block is the reasoning.
- Start at the default. Anthropic's [Opus 4.6 announcement](https://www.anthropic.com/news/claude-opus-4-6), 5 February 2026, said to dial effort down from high to medium when the model overthinks a task.
- Fix a vague goal before you raise the effort. OpenAI's guide calls effort a tuning knob, not the primary way to recover quality.

## A think tool for long chains of tool calls

Extended thinking happens before the reply starts. Anthropic's [think tool post](https://www.anthropic.com/research/claude-think-tool), published 20 March 2025, gave Claude a place to stop and think in the middle of a task. The tool fetches nothing and changes nothing. It only logs a thought. Claude calls it after a tool result to check whether it has what it needs to go on.

The post measured Claude 3.7 Sonnet on the airline tasks of τ-bench, a benchmark of customer service work with tools and policies. The tool alone and extended thinking alone scored about the same, and the best result paired the tool with a prompt that gave examples of how to reason about the airline policy.

On 15 December 2025 Anthropic put a note at the top of the post: extended thinking had improved enough that it now recommends that feature over a dedicated think tool in most cases. The post had already named where the tool is unlikely to help: a single tool call, parallel calls, and simple instruction following.

````json
{
  "name": "think",
  "description": "Use the tool to think about something. It will not obtain new information or change the database, but just append the thought to the log. Use it when complex reasoning or some cache memory is needed.",
  "input_schema": {
    "type": "object",
    "properties": {
      "thought": {
        "type": "string",
        "description": "A thought to think about."
      }
    },
    "required": ["thought"]
  }
}
````

**τ-bench airline, pass^1, Claude 3.7 Sonnet** The share of tasks passed on the first trial, from the post's table. Its text gives 0.370 and 0.570 for the baseline and the gold bar.

- Baseline, no tool and no extended thinking: 0.33
- Extended thinking: 0.41
- Think tool: 0.4
- Think tool with the example prompt: **0.58**

Source: [Anthropic: The "think" tool](https://www.anthropic.com/research/claude-think-tool) (read 2026-11-06)

## Shown reasoning is not always the real reason

In a study published on 3 April 2025, [Anthropic's Alignment Science team](https://www.anthropic.com/research/reasoning-models-dont-say-think) slipped hints about the answer into quiz questions for two reasoning models. It then checked whether each model's chain of thought mentioned a hint the model had used. Most of the time, it did not.

The unfaithful chains were longer on average than the faithful ones. Length is no sign of honesty. The authors call their setup contrived, with multiple-choice questions and planted hints. [What are LLM evals](https://mindket.com/learn/ai-evals/what-are-llm-evals) shows how to grade a final answer with checks of your own. Use the chain of thought as a lead when you debug, not as the grade.

**How often a model mentioned a hint it used**

- **25%** Claude 3.7 Sonnet, on average across hint types (Source: [Anthropic: Reasoning models don't always say what they think](https://www.anthropic.com/research/reasoning-models-dont-say-think), read 2026-11-06)
- **39%** DeepSeek R1, on average across hint types (Source: [Anthropic: Reasoning models don't always say what they think](https://www.anthropic.com/research/reasoning-models-dont-say-think), read 2026-11-06)

## Jason Wei on chain of thought after o1, Alex Albert on the think tool

Jason Wei, then at Google Research, was first author of the [2022 chain-of-thought paper](https://huggingface.co/papers/2201.11903). On 10 November 2024 he [posted on X](https://x.com/_jasonwei/status/1855417833775309171) that there is a "nuanced but important difference" between chain of thought before and after o1. Before the o1 paradigm, he wrote, there was a mismatch between what chain of thought was and what we wanted it to be.

That fits this lesson's advice. On an older model the chain is a format you ask for and can parse. On a reasoning model it is the model's own process, so you set the effort and judge the answer.

Alex Albert of Anthropic [shared the think tool research on X](https://x.com/alexalbert__/status/1903130655564922911) on 21 March 2025. He wrote that a simple think tool improved instruction adherence and multi-step problem solving for agents. The December note above came nine months later.

> There is a nuanced but important difference between chain-of-thought before and after o1.
>
> Jason Wei (@_jasonwei), First author of the 2022 chain-of-thought paper, [X](https://x.com/_jasonwei/status/1855417833775309171) (read 2026-11-06)

[![Jason Wei (@_jasonwei) on X, 10 November 2024: There is a nuanced but important difference between chain-of-thought before and after o1. Before the o1 paradigm, he writes, there was a mismatch between what chain of thought was and what we wanted it to be.](https://mindket.com/blog/shots/x-jasonwei-cot-before-after-o1.webp?v=f91d6533e36f)](https://x.com/_jasonwei/status/1855417833775309171)

Jason Wei on X, 10 November 2024

[![Alex Albert (@alexalbert__) on X, 21 March 2025: New research from our team at @AnthropicAI shows how giving Claude a simple 'think' tool dramatically improves instruction adherence and multi-step problem solving for agents.](https://mindket.com/blog/shots/x-alexalbert-think-tool.webp?v=9f9b174f655d)](https://x.com/alexalbert__/status/1903130655564922911)

Alex Albert on X, 21 March 2025

[Claude 3.7 Sonnet with extended thinking](https://www.youtube.com/watch?v=t3nnDXa81Hs)

Anthropic's launch video for Claude 3.7 Sonnet, 24 February 2025, on its own channel: one model that gives near-instant responses or thinks step by step before it answers.

> Screenshots of X's own embeds of the posts, taken on 6 November 2026, with the time in UTC. X cuts the first post short with Show more. Select a screenshot to open the post on X.

## Common mistakes

- Showing users the brainstorm or the thinking text when they asked for an answer.
- Setting the output token cap so low that OpenAI returns an incomplete response with no visible text, while the input and reasoning are still billed.
- Setting effort to the top level on every call. OpenAI says to use `xhigh` only when your evals show a clear benefit that justifies the extra latency and cost.
- Changing `reasoning.effort` on the request partway through a conversation. On supported GPT-6 and later models, OpenAI's guide says to add a `configuration_update` item between responses and leave the request's effort alone, which preserves the original prompt prefix for prompt caching.
- Reading a long chain of thought as proof that the answer is right.

## Try it yourself

Each exercise changes one setting on a prompt or call from this lesson, so you can see what that setting does. The [next lesson](https://mindket.com/learn/prompt-engineering-advanced/prompt-chaining) splits one task across several prompts instead of one long chain.

- Run the five-talks puzzle at `low` and at `xhigh` effort, then compare `usage.output_tokens` and the orderings each reply lists.
- Send the actor prompt with the brainstorm sentence to a reasoning model. Does the answer change, or only the length?
- Remove `display` from the call and look at what the thinking block holds.
- Add the think tool to an agent you already run, and log every thought it writes across ten tasks.

## Check yourself

1. What does OpenAI advise about "think step by step" prompts for its reasoning models?

   A. Add them to every request
   B. Avoid them, since the models already reason internally
   C. Put them in the developer message only
   D. Pair them with a high effort

   Answer: B. The reasoning best practices call these prompts unnecessary and warn they can sometimes hinder the result.

2. A call to Claude Sonnet 5.5 sends thinking with the type "enabled" and a budget_tokens value. What happens?

   A. Claude thinks up to that budget
   B. Claude ignores the budget
   C. The call falls back to medium effort
   D. The API returns a 400 error

   Answer: D. Anthropic's cookbook says this model does not accept the legacy budget form. The effort setting replaces it.

3. In Anthropic's April 2025 study, how often did Claude 3.7 Sonnet mention a hint it had used?

   A. 25% of the time on average
   B. Every time
   C. Never
   D. Nine times in ten

   Answer: A. DeepSeek R1 mentioned it 39% of the time, so most chains in the study left the hint out.

4. A prompt tells a model without built-in reasoning to think, but to print only the answer. What does Anthropic's tutorial say happens?

   A. It thinks silently first
   B. The reply gets longer
   C. No thinking takes place
   D. The API returns an error

   Answer: C. Thinking only counts when it's out loud, because the reasoning is the text the model writes.

## Frequently asked questions

### What is chain of thought prompting?

It is a prompting method that gets a model to show its working in the reply, where you can read and check each step. Jason Wei and colleagues coined the term in 2022 and found the ability emerging in sufficiently large language models.

### Does "Let's think step by step" still work?

Outside reasoning models, often. OpenAI's archived [GPT-4.1 prompting guide](https://developers.openai.com/cookbook/examples/gpt4-1_prompting_guide) recommends ending the prompt with a basic chain-of-thought instruction, and it reported that prompting GPT-4.1 to plan explicitly raised its SWE-bench Verified pass rate by 4%. The reasoning best practices call the same line unnecessary.

### What is the difference between chain of thought and extended thinking?

Chain of thought is text you ask for, and it arrives in the reply where you can read and parse it. Extended thinking happens in separate thinking blocks before the reply. An effort setting controls it, you see a summary or nothing, and you pay for all of it as output.

### Is chain of thought the same as prompt chaining?

No. One happens inside a single response. Prompt chaining splits a task across several prompts, each fed the last one's output, so your code can check the result between steps.

### Should reasoning carry over to the next turn?

On OpenAI's Responses API, pass `previous_response_id` and earlier reasoning items stay available. GPT-5.6 models render that reasoning into the next turn by default, older models do not, and `reasoning.context` picks either behavior. Anthropic's extended thinking cookbook says redacted thinking blocks are decrypted when passed back to the API, so Claude can continue its response without losing context.

## Sources

- [Hugging Face Papers: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022)](https://huggingface.co/papers/2201.11903) (read 2026-11-06)
- [Hugging Face Papers: Large Language Models are Zero-Shot Reasoners (2022)](https://huggingface.co/papers/2205.11916) (read 2026-11-06)
- [Anthropic on GitHub: Prompt engineering interactive tutorial, chapter 6, Precognition (Thinking Step by Step)](https://github.com/anthropics/prompt-eng-interactive-tutorial/blob/master/Anthropic%201P/06_Precognition_Thinking_Step_by_Step.ipynb) (read 2026-11-06)
- [OpenAI Docs: Reasoning models](https://developers.openai.com/api/docs/guides/reasoning) (read 2026-11-06)
- [OpenAI Docs: Reasoning best practices](https://developers.openai.com/api/docs/guides/reasoning-best-practices) (read 2026-11-06)
- [OpenAI Cookbook: GPT-4.1 Prompting Guide](https://developers.openai.com/cookbook/examples/gpt4-1_prompting_guide) (read 2026-11-06)
- [Anthropic on GitHub: Claude cookbooks, Extended Thinking](https://github.com/anthropics/claude-cookbooks/blob/main/extended_thinking/extended_thinking.ipynb) (read 2026-11-06)
- [Anthropic: Introducing Claude Opus 4.6](https://www.anthropic.com/news/claude-opus-4-6) (read 2026-11-06)
- [Anthropic: The "think" tool](https://www.anthropic.com/research/claude-think-tool) (read 2026-11-06)
- [Anthropic: Reasoning models don't always say what they think](https://www.anthropic.com/research/reasoning-models-dont-say-think) (read 2026-11-06)
- [X: Jason Wei on chain of thought before and after o1](https://x.com/_jasonwei/status/1855417833775309171) (read 2026-11-06)
- [X: Alex Albert on the think tool research](https://x.com/alexalbert__/status/1903130655564922911) (read 2026-11-06)
- [YouTube: Anthropic, Claude 3.7 Sonnet with extended thinking](https://www.youtube.com/watch?v=t3nnDXa81Hs) (read 2026-11-06)

[Browse AI agents](https://mindket.com/agents)