RAG tutorial: build a retrieval pipeline step by step

Before you start

You need Python 3, a Voyage AI key and an Anthropic key. Anthropic offers no embedding model of its own, so its embeddings guide points to Voyage AI, and this lesson does the same. Any embeddings provider fits the pipeline, and lesson 2 compares them.

Check that you need RAG at all. Anthropic's Contextual Retrieval post says a knowledge base under 200,000 tokens, about 500 pages, can go straight into the prompt, and that prompt caching makes this faster and more cost-effective. Retrieval earns its extra moving parts once the corpus no longer fits.

Build the pipeline

The script below is the whole pipeline. The splitter call comes from LangChain's semantic search tutorial, the two embedding calls and the dot product from the Voyage AI quickstart, and the search_result blocks from Anthropic's search results guide. The glue between them, the file names and the question are our sample, so swap in your own.

import numpy as np
import voyageai
import anthropic
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter

vo = voyageai.Client()          # reads VOYAGE_API_KEY
claude = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

# 1. Load: one Document per file, with a source that never changes
docs = [
    Document(
        page_content=open("refund-policy.txt", encoding="utf-8").read(),
        metadata={"source": "kb://refund-policy", "title": "Refund policy"},
    ),
    Document(
        page_content=open("shipping.txt", encoding="utf-8").read(),
        metadata={"source": "kb://shipping", "title": "Shipping"},
    ),
]

# 2. Chunk
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200, add_start_index=True
)
chunks = splitter.split_documents(docs)

# 3. Embed the chunks (index time)
chunk_embeddings = vo.embed(
    texts=[c.page_content for c in chunks],
    model="voyage-4-large",
    input_type="document",
).embeddings

# 4. Store: a plain list in memory is enough for a first run

# 5. Retrieve (query time)
question = "How long do I have to return an item?"
query_embedding = vo.embed(
    texts=[question],
    model="voyage-4-large",
    input_type="query",
).embeddings[0]
similarities = np.dot(chunk_embeddings, query_embedding)
top = np.argsort(-similarities)[:4]

# 6. Generate, with citations on
results = [
    {
        "type": "search_result",
        "source": chunks[i].metadata["source"],
        "title": chunks[i].metadata["title"],
        "content": [{"type": "text", "text": chunks[i].page_content}],
        "citations": {"enabled": True},
    }
    for i in top
]
response = claude.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": [*results, {"type": "text", "text": question}]}
    ],
)

for block in response.content:
    if block.type == "text":
        print(block.text)
        for c in block.citations or []:
            print(f"  [{c.title}] {c.source}")

1. Install the packages with pip install numpy voyageai anthropic langchain-core langchain-text-splitters. Set VOYAGE_API_KEY and ANTHROPIC_API_KEY, which the two clients read from the environment.
2. Load: wrap each file in a LangChain Document. Give it a source that never changes, such as kb://refund-policy, and a title. Both come back inside every citation.
3. Chunk: RecursiveCharacterTextSplitter with chunk_size=1000 and chunk_overlap=200, the values LangChain uses in its tutorials. add_start_index=True keeps each chunk's character offset in its metadata.
4. Embed: send the chunk texts with input_type="document". The model is voyage-4-large, the one in the Voyage quickstart.
5. Store: the vectors stay in a Python list for now. They vanish when the script ends, and LangChain tells you to persist the store in production.
6. Retrieve: embed the question with input_type="query", take the dot product with every chunk vector and keep the four highest scores. Voyage embeddings are normalized to length 1, so the dot product equals cosine similarity.
7. Generate: pass the four chunks as search_result blocks with citations enabled, then the question as a text block in the same user message.
8. Run it. Each text block that draws on a chunk carries citations, which the last loop prints.

Index time and query time

Stages 1 to 4 run when your documents change. Stages 5 and 6 run on every question. The split decides what a fix costs: a change to chunking or to the embedding model means re-embedding the corpus, while a change to the prompt or the number of chunks does not.

The embedding model is the part you cannot change quietly. LlamaIndex's docs say that if you change the embedding model you must re-index your data, and that the same model has to embed both the documents and the queries. Voyage AI documents one exception. Its model page says all embeddings from the 4 series are compatible with each other, which lesson 2 puts to use.

What a citation contains

If an older tutorial sends the search-results-2025-06-09 beta header, drop it. Search result blocks have needed no beta header on the Claude API since 8 August 2025.

Citations are off by default, and every search result in one request must use the same setting. When they are on, no special prompting is needed. Each text block in the reply that uses your content carries a search_result_location citation with these fields:

• source and title: copied from the search result you sent.
• cited_text: the full text of the cited block or blocks. It does not count toward output tokens.
• search_result_index: the 0-based position of the result among all search_result blocks in the request.
• start_block_index and end_block_index: a slice of the result's content array, the end exclusive.

The text block is the smallest unit Claude can cite, and it cites whole blocks, not substrings. To get sentence-level citations, put each sentence in its own block inside content. If your prompt asks for JSON only, expect few or no citations.

Two-step RAG or an agent that decides

What you built is two-step RAG, LangChain's name for retrieval that always runs before generation. At the other end is agentic RAG: you expose search as a tool, and the model decides whether to call it and with which query. Anthropic's search results guide covers both shapes. Results can sit in the user message, as above, or come back from a tool call, and tool_choice can force that call so retrieval runs every time.

We'd begin with two-step. Its latency is easier to predict because the number of model calls is fixed, and a bad answer has only two places to look. Move to an agent when single questions need several lookups. A retrieval function wrapped in an MCP server is one way to hand it to one, and what an AI agent is covers the loop around the tool.

Alex Albert on cited search results, Hamel Husain on long context

Alex Albert, who works in research at Anthropic according to his X bio, announced the search_result blocks on X on 3 July 2025, the day they shipped. His thread put the mechanics in one line. You give Claude a source, a title and the content, and each citation names the source and the text it used. In your script, the two metadata fields from step 2 are then the only link between an answer and a file. Pick values a person can follow, such as a URL or a page ID that never changes.

Hamel Husain, who works on evals at OpenAI by his X bio, answered the claim that long context windows make RAG irrelevant on X on 10 July 2024. He gave three reasons. You often filter by metadata such as dates. Retrieval increasingly decides which sources to pull from. Dropping clearly irrelevant documents gives better results however long the window is.

A small corpus can skip retrieval under Anthropic's size rule from Before you start, but filters bring it back. Once users should only see their own tickets or this year's policy, you need a retrieval step with a WHERE clause, and lesson 2 adds one in pgvector.

Screenshots of X's own embeds of the posts, taken on 5 November 2026, with the time in UTC. X cuts Hamel Husain's post short with Show more. Select one to open the post on X.

Common mistakes

• Leaving out input_type. Voyage's guide says never to omit it for retrieval, since it adds a retrieval prompt in front of queries and documents.
• Embedding the whole corpus in one call. Voyage's embed takes at most 1,000 texts per request and caps the total tokens by model, 120K for voyage-4-large. Send the chunks in batches, as the text embeddings page limits require.
• Embedding documents with one model and queries with an unrelated one. The vectors only compare when both come from the same model, or from models the vendor says share one embedding space.
• Sending more chunks because more feels safer. Anthropic's search results guide advises returning only the most relevant results to avoid context overflow. The Contextual Retrieval post warns that extra information can distract the model, so there is a limit. Measure yours in lesson 4.
• Treating retrieved text as trusted. LangChain warns that RAG apps are open to indirect prompt injection, because retrieved chunks share the context window with your system prompt. No prompt or delimiter strategy fully prevents it, so validate outputs before users see them.
• Dropping the source. A search result's source can be any stable string, a URL or an ID such as kb://article-1234, but it has to exist for a citation to point anywhere.

Try it yourself

• Ask something your documents do not answer. Check whether Claude says so or fills the gap from memory.
• Print similarities[top] and look for the score where relevant chunks give way to noise.
• Set chunk_size to 500 and chunk_overlap to 50, then raise the four chunks to eight. LlamaIndex advises raising the top-k when you shrink the chunks.
• Split one chunk into sentence-sized blocks inside a single search_result and compare cited_text before and after.

Next

Lesson 2 opens the embed and store stages: what a vector is, how to pick a model and a dimension, and how to move the in-memory list into Postgres with pgvector. The prompt budget behind the four-chunk limit is the subject of the context window lesson, and what are tokens in AI explains how it is counted.

Frequently asked questions

Sources

1. LangChain Docs: Retrieval Augmented Generation (RAG) with Deep Agents (read November 4, 2026)https://docs.langchain.com/oss/python/deepagents/rag
2. LangChain Docs: Retrieval (read November 4, 2026)https://docs.langchain.com/oss/python/langchain/retrieval
3. LangChain Docs: Build a semantic search engine (read November 4, 2026)https://docs.langchain.com/oss/python/langchain/knowledge-base
4. Claude Docs: Embeddings (read November 4, 2026)https://platform.claude.com/docs/en/build-with-claude/embeddings
5. Claude Docs: Search results (read November 5, 2026)https://platform.claude.com/docs/en/build-with-claude/search-results
6. Claude Docs: Release notes (read November 5, 2026)https://platform.claude.com/docs/en/release-notes/overview
7. Anthropic: Contextual Retrieval in AI systems (read November 4, 2026)https://www.anthropic.com/research/contextual-retrieval
8. Voyage AI Docs: Quick Start (read November 5, 2026)https://www.mongodb.com/docs/voyageai/quickstart
9. Voyage AI Docs: Text Embeddings (read November 5, 2026)https://www.mongodb.com/docs/voyageai/models/text-embeddings
10. LlamaIndex Docs: Basic strategies (read November 4, 2026)https://developers.llamaindex.ai/python/framework/optimizing/basic_strategies/basic_strategies/
11. Alex Albert on X: Search result content blocks (3 July 2025) (read November 5, 2026)https://x.com/alexalbert__/status/1940823684362781039
12. Alex Albert on X: how search result blocks work (3 July 2025) (read November 5, 2026)https://x.com/alexalbert__/status/1940823696236859855
13. Hamel Husain on X: RAG and long context windows (10 July 2024) (read November 5, 2026)https://x.com/HamelHusain/status/1811050329972322708
14. freeCodeCamp.org on YouTube: Learn RAG From Scratch (read November 5, 2026)https://www.youtube.com/watch?v=sVcwVQRHIc8