# RAG tutorial: build a retrieval pipeline step by step

> Retrieval-augmented generation (RAG) looks up passages from your own documents when a question arrives and puts them in the model's prompt. A minimal pipeline has six stages: load, chunk, embed, store, retrieve and generate. This tutorial builds all six in one Python script and has Claude cite the passages it used.

Mindket editors · Published 2026-11-04 · Updated 2026-11-05 · https://mindket.com/learn/rag/rag-tutorial

## In short

- Build all six stages of a RAG pipeline in one Python script.
- Tell index time from query time, and know which settings live in each.
- Get answers that cite your own source IDs, down to the text block.
- Choose between two-step RAG and an agent that decides when to retrieve.

## Before you start

You need Python 3, a [Voyage AI key](https://dash.voyageai.com/) and an [Anthropic key](https://platform.claude.com/settings/keys). Anthropic offers no embedding model of its own, so its [embeddings guide](https://platform.claude.com/docs/en/build-with-claude/embeddings) points to Voyage AI, and this lesson does the same. Any embeddings provider fits the pipeline, and [lesson 2](https://mindket.com/learn/rag/embeddings-explained) compares them.

Check that you need RAG at all. Anthropic's [Contextual Retrieval post](https://www.anthropic.com/research/contextual-retrieval) says a knowledge base under 200,000 tokens, about 500 pages, can go straight into the prompt, and that prompt caching makes this faster and more cost-effective. Retrieval earns its extra moving parts once the corpus no longer fits.

## Build the pipeline

The script below is the whole pipeline. The splitter call comes from LangChain's [semantic search tutorial](https://docs.langchain.com/oss/python/langchain/knowledge-base), the two embedding calls and the dot product from the Voyage AI [quickstart](https://www.mongodb.com/docs/voyageai/quickstart), and the `search_result` blocks from Anthropic's [search results guide](https://platform.claude.com/docs/en/build-with-claude/search-results). The glue between them, the file names and the question are our sample, so swap in your own.

````python
import numpy as np
import voyageai
import anthropic
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter

vo = voyageai.Client()          # reads VOYAGE_API_KEY
claude = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

# 1. Load: one Document per file, with a source that never changes
docs = [
    Document(
        page_content=open("refund-policy.txt", encoding="utf-8").read(),
        metadata={"source": "kb://refund-policy", "title": "Refund policy"},
    ),
    Document(
        page_content=open("shipping.txt", encoding="utf-8").read(),
        metadata={"source": "kb://shipping", "title": "Shipping"},
    ),
]

# 2. Chunk
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200, add_start_index=True
)
chunks = splitter.split_documents(docs)

# 3. Embed the chunks (index time)
chunk_embeddings = vo.embed(
    texts=[c.page_content for c in chunks],
    model="voyage-4-large",
    input_type="document",
).embeddings

# 4. Store: a plain list in memory is enough for a first run

# 5. Retrieve (query time)
question = "How long do I have to return an item?"
query_embedding = vo.embed(
    texts=[question],
    model="voyage-4-large",
    input_type="query",
).embeddings[0]
similarities = np.dot(chunk_embeddings, query_embedding)
top = np.argsort(-similarities)[:4]

# 6. Generate, with citations on
results = [
    {
        "type": "search_result",
        "source": chunks[i].metadata["source"],
        "title": chunks[i].metadata["title"],
        "content": [{"type": "text", "text": chunks[i].page_content}],
        "citations": {"enabled": True},
    }
    for i in top
]
response = claude.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": [*results, {"type": "text", "text": question}]}
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)
        for c in block.citations or []:
            print(f"  [{c.title}] {c.source}")
````

1. Install the packages with `pip install numpy voyageai anthropic langchain-core langchain-text-splitters`. Set `VOYAGE_API_KEY` and `ANTHROPIC_API_KEY`, which the two clients read from the environment.
2. Load: wrap each file in a LangChain `Document`. Give it a `source` that never changes, such as `kb://refund-policy`, and a `title`. Both come back inside every citation.
3. Chunk: `RecursiveCharacterTextSplitter` with `chunk_size=1000` and `chunk_overlap=200`, the values LangChain uses in its tutorials. `add_start_index=True` keeps each chunk's character offset in its metadata.
4. Embed: send the chunk texts with `input_type="document"`. The model is `voyage-4-large`, the one in the Voyage quickstart.
5. Store: the vectors stay in a Python list for now. They vanish when the script ends, and LangChain tells you to persist the store in production.
6. Retrieve: embed the question with `input_type="query"`, take the dot product with every chunk vector and keep the four highest scores. Voyage embeddings are normalized to length 1, so the dot product equals cosine similarity.
7. Generate: pass the four chunks as `search_result` blocks with `citations` enabled, then the question as a text block in the same user message.
8. Run it. Each text block that draws on a chunk carries citations, which the last loop prints.

## Index time and query time

Stages 1 to 4 run when your documents change. Stages 5 and 6 run on every question. The split decides what a fix costs: a change to chunking or to the embedding model means re-embedding the corpus, while a change to the prompt or the number of chunks does not.

The embedding model is the part you cannot change quietly. [LlamaIndex's docs](https://developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/) say that if you change the embedding model you must re-index your data, and that the same model has to embed both the documents and the queries. Voyage AI documents one exception. Its [model page](https://www.mongodb.com/docs/voyageai/models/text-embeddings) says all embeddings from the 4 series are compatible with each other, which lesson 2 puts to use.

| Stage | Runs when | Settings to know | Covered in |
| --- | --- | --- | --- |
| Load | Documents change | The loader, the source and title you attach | This lesson |
| Chunk | Documents change | chunk_size, chunk_overlap, the splitter type | Lesson 3 |
| Embed | Documents change | model, input_type, output dimension | Lesson 2 |
| Store | Documents change | Index type and its search settings | Lesson 2 |
| Retrieve | Every question | Number of chunks, hybrid search, reranking | Lessons 3 and 4 |
| Generate | Every question | The prompt, the citation settings | This lesson |

## What a citation contains

If an older tutorial sends the `search-results-2025-06-09` beta header, drop it. Search result blocks have needed no beta header on the Claude API since 8 August 2025.

Citations are off by default, and every search result in one request must use the same setting. When they are on, no special prompting is needed. Each text block in the reply that uses your content carries a `search_result_location` citation with these fields:

- `source` and `title`: copied from the search result you sent.
- `cited_text`: the full text of the cited block or blocks. It does not count toward output tokens.
- `search_result_index`: the 0-based position of the result among all `search_result` blocks in the request.
- `start_block_index` and `end_block_index`: a slice of the result's `content` array, the end exclusive.

**How Claude's citations for your own content arrived**

1. **January 23, 2025** Citations in the API. Claude can attribute what it says to plain text and custom content documents you send.
2. **July 3, 2025** Search result blocks, in beta. Your own tools can return search results that Claude cites, behind the search-results-2025-06-09 header.
3. **August 8, 2025** Search result blocks leave beta. No beta header on the Claude API and Vertex AI, which is why the script sends none.
4. **September 3, 2025** Citable documents in tool results. A client-side tool can return documents that Claude cites, the shape agentic RAG uses.
5. **November 21, 2025** Search result blocks on Amazon Bedrock. Available there too, again with no beta header.

Source: [Claude Docs: Release notes](https://platform.claude.com/docs/en/release-notes/overview) (read 2026-11-05)

> The text block is the smallest unit Claude can cite, and it cites whole blocks, not substrings. To get sentence-level citations, put each sentence in its own block inside `content`. If your prompt asks for JSON only, expect few or no citations.

## Two-step RAG or an agent that decides

What you built is two-step RAG, LangChain's name for retrieval that always runs before generation. At the other end is agentic RAG: you expose search as a tool, and the model decides whether to call it and with which query. Anthropic's search results guide covers both shapes. Results can sit in the user message, as above, or come back from a tool call, and `tool_choice` can force that call so retrieval runs every time.

We'd begin with two-step. Its latency is easier to predict because the number of model calls is fixed, and a bad answer has only two places to look. Move to an agent when single questions need several lookups. A retrieval function wrapped in an [MCP server](https://mindket.com/learn/mcp/build-mcp-server-python) is one way to hand it to one, and [what an AI agent is](https://mindket.com/learn/prompt-engineering/what-is-an-ai-agent) covers the loop around the tool.

| Architecture | Who decides to retrieve | Latency | Fits |
| --- | --- | --- | --- |
| 2-step RAG | Your code, on every question | Fast, predictable | FAQs, documentation bots |
| Agentic RAG | The model, during its reasoning | Variable | Research assistants with several tools |
| Hybrid | Both, with validation steps | Variable | Domain Q&A that needs quality checks |

## Alex Albert on cited search results, Hamel Husain on long context

Alex Albert, who works in research at Anthropic according to his X bio, announced the `search_result` blocks [on X](https://x.com/alexalbert__/status/1940823684362781039) on 3 July 2025, the day they shipped. His thread put the mechanics in one line. You give Claude a source, a title and the content, and each citation names the source and the text it used. In your script, the two metadata fields from step 2 are then the only link between an answer and a file. Pick values a person can follow, such as a URL or a page ID that never changes.

Hamel Husain, who works on evals at OpenAI by his X bio, answered the claim that long context windows make RAG irrelevant [on X](https://x.com/HamelHusain/status/1811050329972322708) on 10 July 2024. He gave three reasons. You often filter by metadata such as dates. Retrieval increasingly decides which sources to pull from. Dropping clearly irrelevant documents gives better results however long the window is.

A small corpus can skip retrieval under Anthropic's size rule from Before you start, but filters bring it back. Once users should only see their own tickets or this year's policy, you need a retrieval step with a `WHERE` clause, and lesson 2 adds one in pgvector.

> You are gonna get better results by eliminating clearly irrelevant documents no matter how long the context window is
>
> Hamel Husain, Works on evals at OpenAI, by his X bio, [X](https://x.com/HamelHusain/status/1811050329972322708) (read 2026-11-05)

[![Alex Albert (@alexalbert__) on X, 3 July 2025: We shipped a new feature in the Anthropic API today: Search result content blocks. This enables natural citations with proper source attribution for all RAG content, bringing web search-quality citations to your custom search tools.](https://mindket.com/blog/shots/x-alexalbert-search-result-blocks.webp?v=06e3ef6955ee)](https://x.com/alexalbert__/status/1940823684362781039)

Alex Albert on X, 3 July 2025

[![Hamel Husain (@HamelHusain) on X, 10 July 2024, disagreeing with takes that long context windows will make RAG irrelevant. His first reason: You often want to filter by metadata (dates, # of views, etc). Search/IR is critical in many applied scenarios.](https://mindket.com/blog/shots/x-hamelhusain-rag-long-context.webp?v=a30397a23dc7)](https://x.com/HamelHusain/status/1811050329972322708)

Hamel Husain on X, 10 July 2024

[Learn RAG From Scratch: Python AI Tutorial from a LangChain Engineer](https://www.youtube.com/watch?v=sVcwVQRHIc8)

freeCodeCamp.org on YouTube: a course on building RAG from scratch in Python, taught by a LangChain engineer, as its title says.

> Screenshots of X's own embeds of the posts, taken on 5 November 2026, with the time in UTC. X cuts Hamel Husain's post short with Show more. Select one to open the post on X.

## Common mistakes

- Leaving out `input_type`. Voyage's guide says never to omit it for retrieval, since it adds a retrieval prompt in front of queries and documents.
- Embedding the whole corpus in one call. Voyage's `embed` takes at most 1,000 texts per request and caps the total tokens by model, 120K for `voyage-4-large`. Send the chunks in batches, as the [text embeddings page](https://www.mongodb.com/docs/voyageai/models/text-embeddings) limits require.
- Embedding documents with one model and queries with an unrelated one. The vectors only compare when both come from the same model, or from models the vendor says share one embedding space.
- Sending more chunks because more feels safer. Anthropic's search results guide advises returning only the most relevant results to avoid context overflow. The Contextual Retrieval post warns that extra information can distract the model, so there is a limit. Measure yours in lesson 4.
- Treating retrieved text as trusted. [LangChain warns](https://docs.langchain.com/oss/python/deepagents/rag) that RAG apps are open to indirect prompt injection, because retrieved chunks share the context window with your system prompt. No prompt or delimiter strategy fully prevents it, so validate outputs before users see them.
- Dropping the source. A search result's `source` can be any stable string, a URL or an ID such as `kb://article-1234`, but it has to exist for a citation to point anywhere.

## Try it yourself

- Ask something your documents do not answer. Check whether Claude says so or fills the gap from memory.
- Print `similarities[top]` and look for the score where relevant chunks give way to noise.
- Set `chunk_size` to 500 and `chunk_overlap` to 50, then raise the four chunks to eight. LlamaIndex advises raising the top-k when you shrink the chunks.
- Split one chunk into sentence-sized blocks inside a single `search_result` and compare `cited_text` before and after.

## Next

[Lesson 2](https://mindket.com/learn/rag/embeddings-explained) opens the embed and store stages: what a vector is, how to pick a model and a dimension, and how to move the in-memory list into Postgres with pgvector. The prompt budget behind the four-chunk limit is the subject of [the context window lesson](https://mindket.com/learn/prompt-engineering/context-window), and [what are tokens in AI](https://mindket.com/blog/what-are-tokens-in-ai) explains how it is counted.

## Check yourself

1. Which input type do you set when you embed the user's question for retrieval?

   A. document
   B. query
   C. search
   D. None, the default

   Answer: B. Voyage's guide says to use query for questions and document for the texts you index, and not to omit it for retrieval.

2. Your whole knowledge base is 150,000 tokens. What does the Contextual Retrieval post suggest?

   A. Build a vector index first
   B. Split it into small chunks and rerank them
   C. Put it all in the prompt and cache it
   D. Fine-tune a model on it

   Answer: C. The post says a knowledge base under 200,000 tokens can go straight into the prompt, and prompt caching makes that faster and more cost-effective.

3. You pass each chunk as one text block in a search result. What is the finest citation Claude can give?

   A. The whole chunk
   B. One sentence inside it
   C. A character range
   D. Only the document title

   Answer: A. The text block is the smallest citable unit, so Claude cites whole blocks. Split a chunk into smaller blocks for finer citations.

4. You move from OpenAI's text-embedding-3-small to voyage-4-large. What should you do with the stored vectors?

   A. Keep them, vectors carry over
   B. Re-embed only the new documents
   C. Normalize the old vectors again
   D. Re-embed every chunk with the new model

   Answer: D. LlamaIndex says a changed embedding model means re-indexing your data. Only models whose vendor says they share one embedding space, such as the Voyage 4 series, skip that.

## Frequently asked questions

### What is RAG in simple terms?

Your code searches your documents first and hands the best matches to the model, so it answers from current, private data and not from memory alone. LangChain's retrieval overview names two limits it fixes: finite context and static knowledge.

### Do I need a vector database to build RAG?

No. The pipeline above keeps its vectors in a Python list and ranks them with a NumPy dot product. A database starts to matter when the vectors must persist, the corpus grows or you need filters. Lesson 2 moves them into Postgres with pgvector.

### Is RAG better than putting the documents in the prompt?

Only once the corpus outgrows the context window, or once users need filters such as dates or permissions. Below that, sending everything with prompt caching is the simpler build to run and debug.

### Do I need LangChain or LlamaIndex for RAG?

No. LangChain supplies only the splitter here, and everything else is plain API calls. LangChain's own docs call its RecursiveCharacterTextSplitter the recommended splitter for generic text.

### How many chunks should I send to the model?

Start with four and measure. Anthropic tested 5, 10 and 20 chunks in its Contextual Retrieval post and found 20 the most effective of the three on its data, so the right number depends on your documents.

### How does Claude cite RAG sources?

Through search_result blocks with citations enabled. Each cited text block in the reply carries the source, the title and the range of content blocks it used.

### Does RAG stop hallucinations?

No. It gives the model evidence, and the model can still say more than the evidence supports. Lesson 4 measures that gap with a faithfulness score.

## Sources

- [LangChain Docs: Retrieval Augmented Generation (RAG) with Deep Agents](https://docs.langchain.com/oss/python/deepagents/rag) (read 2026-11-04)
- [LangChain Docs: Retrieval](https://docs.langchain.com/oss/python/langchain/retrieval) (read 2026-11-04)
- [LangChain Docs: Build a semantic search engine](https://docs.langchain.com/oss/python/langchain/knowledge-base) (read 2026-11-04)
- [Claude Docs: Embeddings](https://platform.claude.com/docs/en/build-with-claude/embeddings) (read 2026-11-04)
- [Claude Docs: Search results](https://platform.claude.com/docs/en/build-with-claude/search-results) (read 2026-11-05)
- [Claude Docs: Release notes](https://platform.claude.com/docs/en/release-notes/overview) (read 2026-11-05)
- [Anthropic: Contextual Retrieval in AI systems](https://www.anthropic.com/research/contextual-retrieval) (read 2026-11-04)
- [Voyage AI Docs: Quick Start](https://www.mongodb.com/docs/voyageai/quickstart) (read 2026-11-05)
- [Voyage AI Docs: Text Embeddings](https://www.mongodb.com/docs/voyageai/models/text-embeddings) (read 2026-11-05)
- [LlamaIndex Docs: Basic strategies](https://developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/) (read 2026-11-04)
- [Alex Albert on X: Search result content blocks (3 July 2025)](https://x.com/alexalbert__/status/1940823684362781039) (read 2026-11-05)
- [Alex Albert on X: how search result blocks work (3 July 2025)](https://x.com/alexalbert__/status/1940823696236859855) (read 2026-11-05)
- [Hamel Husain on X: RAG and long context windows (10 July 2024)](https://x.com/HamelHusain/status/1811050329972322708) (read 2026-11-05)
- [freeCodeCamp.org on YouTube: Learn RAG From Scratch](https://www.youtube.com/watch?v=sVcwVQRHIc8) (read 2026-11-05)

[Browse MCP servers](https://mindket.com/mcp-servers)