Chain of thought prompting: when to ask for steps and when to set the effort

Before you start

A chain of thought is the reasoning a model writes before its answer. The 2022 paper that named it, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, defined it as a series of intermediate reasoning steps.

Reasoning models produce that chain without being asked. OpenAI's reasoning models, such as GPT-5.5, and Claude with extended thinking both reason before they reply, and choosing an AI model explains how they differ from fast models. You need an API key for the exercises and the code.

Ask a fast model to think out loud

Chapter 6 of Anthropic's interactive prompting tutorial asks Claude to name a famous movie starring an actor born in 1956. The tutorial uses it as a case where the direct answer comes out wrong. Its fix is one extra sentence: brainstorm some actors and their birth years in a brainstorm tag, then answer.

The tutorial adds a rule worth keeping. Thinking only counts when it's out loud. A prompt that asks the model to think but print only the answer gets no thinking at all, because the reasoning is the text the model writes.

# Prompt 1
Name a famous movie starring an actor who was born in the year 1956.

# Prompt 2
Name a famous movie starring an actor who was born in the year 1956. First brainstorm about some actors and their birth years in <brainstorm> tags, then give your answer.

1. Pick a model that answers without reasoning first, such as GPT-5.5 with the reasoning effort set to none.
2. Send prompt 1 and write down the film, the actor and the birth year the reply gives.
3. Look up that birth year yourself.
4. Send prompt 2 in a fresh conversation. The reply should list actors and years inside the tag before it names a film.
5. In your code, keep only the text after the closing tag, so people see the answer and not the brainstorm.

Why spelled-out steps helped models in 2022

The original paper put the reasoning into few-shot examples, the technique from the previous lesson. Each example showed a question, the worked steps and the answer. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a benchmark of math word problems, ahead of a fine-tuned GPT-3 with a verifier.

Later in 2022, Large Language Models are Zero-Shot Reasoners dropped the examples. Adding "Let's think step by step" before each answer lifted InstructGPT's text-davinci-002 on the same kind of problems, as the chart shows.

Both papers measured models that answered straight away unless prompted to reason, so the chart shows what one sentence did for such a model in 2022. It predicts nothing about today's models. The basics lesson explains why current models rarely need the line.

What changes on a reasoning model

OpenAI's reasoning guide describes reasoning tokens, which the model uses to break down the prompt and weigh approaches before the visible answer. The API never returns them. They still take room in the context window and are billed as output tokens.

Since the model reasons anyway, the reasoning best practices say to avoid chain-of-thought prompts. Telling it to think step by step or explain its reasoning is unnecessary, and the guide warns it may not help and can sometimes hinder. What OpenAI wants instead is a clear goal, strong constraints and an explicit output contract, without every intermediate step written out.

How much the model thinks is a setting on the request. OpenAI calls it the reasoning effort. The guide's example asks for a small bash script at low effort.

from openai import OpenAI

client = OpenAI()

prompt = """
Write a bash script that takes a matrix represented as a string with
format '[1,2],[3,4],[5,6]' and prints the transpose in the same format.
"""

response = client.responses.create(
    model="gpt-6-astra",
    reasoning={"effort": "low"},
    input=[{"role": "user", "content": prompt}],
)

print(response.output_text)

Set the effort on Claude

Anthropic's extended thinking cookbook, written for Claude Sonnet 5.5, sends a scheduling puzzle: five talks, five slots and five constraints. Thinking there is adaptive. Claude decides per request how much to think, and the effort setting steers it. The older form with a fixed budget_tokens returns a 400 error on this model.

The call below is the cookbook's. Only the print loop is this lesson's own, standing in for the cookbook's helper function.

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=16000,  # thinking tokens count toward max_tokens, so leave headroom
    thinking={
        "type": "adaptive",
        "display": "summarized",  # default is "omitted" (empty thinking text)
    },
    output_config={"effort": "xhigh"},  # low | medium | high (default) | xhigh | max
    messages=[
        {
            "role": "user",
            "content": "Five talks (A, B, C, D, E) must be scheduled in five consecutive slots. Constraints: A comes before C; B is not adjacent to D; E is neither first nor last; C is immediately before or after E; D comes before A. List every valid ordering and explain how you know there are no others.",
        }
    ],
)

for block in response.content:
    if block.type == "thinking":
        print("THINKING (summary):", block.thinking)
    elif block.type == "text":
        print("ANSWER:", block.text)

• The thinking text is a summary, yet the bill covers the full reasoning. Leave display out and the blocks come back empty, still billed.
• Easy prompts and low effort can return no thinking block at all, so never assume the first block is the reasoning.
• Start at the default. Anthropic's Opus 4.6 announcement, 5 February 2026, said to dial effort down from high to medium when the model overthinks a task.
• Fix a vague goal before you raise the effort. OpenAI's guide calls effort a tuning knob, not the primary way to recover quality.

A think tool for long chains of tool calls

Extended thinking happens before the reply starts. Anthropic's think tool post, published 20 March 2025, gave Claude a place to stop and think in the middle of a task. The tool fetches nothing and changes nothing. It only logs a thought. Claude calls it after a tool result to check whether it has what it needs to go on.

The post measured Claude 3.7 Sonnet on the airline tasks of τ-bench, a benchmark of customer service work with tools and policies. The tool alone and extended thinking alone scored about the same, and the best result paired the tool with a prompt that gave examples of how to reason about the airline policy.

On 15 December 2025 Anthropic put a note at the top of the post: extended thinking had improved enough that it now recommends that feature over a dedicated think tool in most cases. The post had already named where the tool is unlikely to help: a single tool call, parallel calls, and simple instruction following.

{
  "name": "think",
  "description": "Use the tool to think about something. It will not obtain new information or change the database, but just append the thought to the log. Use it when complex reasoning or some cache memory is needed.",
  "input_schema": {
    "type": "object",
    "properties": {
      "thought": {
        "type": "string",
        "description": "A thought to think about."
      }
    },
    "required": ["thought"]
  }
}

Shown reasoning is not always the real reason

In a study published on 3 April 2025, Anthropic's Alignment Science team slipped hints about the answer into quiz questions for two reasoning models. It then checked whether each model's chain of thought mentioned a hint the model had used. Most of the time, it did not.

The unfaithful chains were longer on average than the faithful ones. Length is no sign of honesty. The authors call their setup contrived, with multiple-choice questions and planted hints. What are LLM evals shows how to grade a final answer with checks of your own. Use the chain of thought as a lead when you debug, not as the grade.

Jason Wei on chain of thought after o1, Alex Albert on the think tool

Jason Wei, then at Google Research, was first author of the 2022 chain-of-thought paper. On 10 November 2024 he posted on X that there is a "nuanced but important difference" between chain of thought before and after o1. Before the o1 paradigm, he wrote, there was a mismatch between what chain of thought was and what we wanted it to be.

That fits this lesson's advice. On an older model the chain is a format you ask for and can parse. On a reasoning model it is the model's own process, so you set the effort and judge the answer.

Alex Albert of Anthropic shared the think tool research on X on 21 March 2025. He wrote that a simple think tool improved instruction adherence and multi-step problem solving for agents. The December note above came nine months later.

Screenshots of X's own embeds of the posts, taken on 6 November 2026, with the time in UTC. X cuts the first post short with Show more. Select a screenshot to open the post on X.

Common mistakes

• Showing users the brainstorm or the thinking text when they asked for an answer.
• Setting the output token cap so low that OpenAI returns an incomplete response with no visible text, while the input and reasoning are still billed.
• Setting effort to the top level on every call. OpenAI says to use xhigh only when your evals show a clear benefit that justifies the extra latency and cost.
• Changing reasoning.effort on the request partway through a conversation. On supported GPT-6 and later models, OpenAI's guide says to add a configuration_update item between responses and leave the request's effort alone, which preserves the original prompt prefix for prompt caching.
• Reading a long chain of thought as proof that the answer is right.

Try it yourself

Each exercise changes one setting on a prompt or call from this lesson, so you can see what that setting does. The next lesson splits one task across several prompts instead of one long chain.

• Run the five-talks puzzle at low and at xhigh effort, then compare usage.output_tokens and the orderings each reply lists.
• Send the actor prompt with the brainstorm sentence to a reasoning model. Does the answer change, or only the length?
• Remove display from the call and look at what the thinking block holds.
• Add the think tool to an agent you already run, and log every thought it writes across ten tasks.

Frequently asked questions

Sources

1. Hugging Face Papers: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022) (read November 6, 2026)https://huggingface.co/papers/2201.11903
2. Hugging Face Papers: Large Language Models are Zero-Shot Reasoners (2022) (read November 6, 2026)https://huggingface.co/papers/2205.11916
3. Anthropic on GitHub: Prompt engineering interactive tutorial, chapter 6, Precognition (Thinking Step by Step) (read November 6, 2026)https://github.com/anthropics/prompt-eng-interactive-tutorial/blob/master/Anthropic%201P/06_Precognition_Thinking_Step_by_Step.ipynb
4. OpenAI Docs: Reasoning models (read November 6, 2026)https://developers.openai.com/api/docs/guides/reasoning
5. OpenAI Docs: Reasoning best practices (read November 6, 2026)https://developers.openai.com/api/docs/guides/reasoning-best-practices
6. OpenAI Cookbook: GPT-4.1 Prompting Guide (read November 6, 2026)https://developers.openai.com/cookbook/examples/gpt4-1_prompting_guide
7. Anthropic on GitHub: Claude cookbooks, Extended Thinking (read November 6, 2026)https://github.com/anthropics/claude-cookbooks/blob/main/extended_thinking/extended_thinking.ipynb
8. Anthropic: Introducing Claude Opus 4.6 (read November 6, 2026)https://www.anthropic.com/news/claude-opus-4-6
9. Anthropic: The "think" tool (read November 6, 2026)https://www.anthropic.com/research/claude-think-tool
10. Anthropic: Reasoning models don't always say what they think (read November 6, 2026)https://www.anthropic.com/research/reasoning-models-dont-say-think
11. X: Jason Wei on chain of thought before and after o1 (read November 6, 2026)https://x.com/_jasonwei/status/1855417833775309171
12. X: Alex Albert on the think tool research (read November 6, 2026)https://x.com/alexalbert__/status/1903130655564922911
13. YouTube: Anthropic, Claude 3.7 Sonnet with extended thinking (read November 6, 2026)https://www.youtube.com/watch?v=t3nnDXa81Hs