Sentence-by-sentence fact check with local Ollama models

Sentence-by-sentence fact check with local Ollama models

An n8n workflow that checks a text against a source document, one sentence at a time. A small fact-checking model served by
Ollama marks each sentence Yes or No, the sentences marked No are collected, and a second Ollama model writes a short summary of
the problems. It can be run by hand or called from another workflow.

What is inside

• workflow.json: the workflow as n8n exports it, thirteen working nodes and five sticky notes. It holds no credential.
• README.md: these notes.

What we changed from the original copy

• The two Ollama nodes were linked to a credential in the author's n8n. The links are gone, so add your own.
• The sample in Edit Fields was two paragraphs about a named researcher, followed by a made-up article about her that mixed in
  invented claims. It is now a short sample about a fictional library, with a document of five sentences and a text of six that
  holds two plainly wrong sentences.
• Nothing else. The Code node is the file's own.

Before you start

• An n8n instance that can reach your Ollama over the network.
• Ollama installed and running, with two models pulled: ollama pull bespoke-minicheck for the checker and
  ollama pull qwen2.5:1.5b for the summary. Ollama's page for the checker, read on 12 November 2026, lists it at 4.7GB with a
  32K context window.

Set it up

1. In n8n, create a workflow, open its menu and choose Import from File. Pick workflow.json.
2. Open Ollama Chat Model and create an Ollama credential. Its Base URL starts as http://localhost:11434. Change it if Ollama
   runs on another machine. If n8n runs in Docker, the container must be able to reach the host's network.
3. Open Ollama Model and select the same credential.
4. Open Edit Fields. The facts field holds the source document and text holds the text to check. Replace both samples.
5. Save and press Execute workflow. The summary is the text field of Basic LLM Chain.

How a run goes

1. A run starts at When clicking ‘Test workflow’, which feeds Edit Fields, or at When Executed by Another Workflow, which
   takes facts and text from the calling workflow.
2. Code splits text into sentences. It stops with "Input text is empty" when the text is blank.
3. Merge1 joins the item that carries facts and text with the item that carries the sentence list, by position.
   Split Out1 then turns the list into one item per sentence, in a field named claim.
4. Basic LLM Chain4 sends each claim to Ollama Chat Model (bespoke-minicheck:latest) as Document: followed by the
   document and Claim: followed by the sentence. The model answers Yes or No.
5. Merge joins each claim with its answer by position. Filter keeps the items whose answer is exactly No.
6. Aggregate puts the kept items in one list. Basic LLM Chain asks Ollama Model (qwen2.5:1.5b) for a summary with
   a system message that sets out the format: a count of incorrect statements, a list of them and a short final assessment.

Change the look

• The document and the text: facts and text in Edit Fields, or the inputs of the calling workflow.
• The models: Model in Ollama Chat Model and in Ollama Model.
• The summary: the system message in Basic LLM Chain. The sticky note says you may delete the summary part and return the raw
  list of wrong sentences from Aggregate.
• What counts as a problem: the No in Filter.
• How the text is split: the regular expression in Code.

Good to know

• The checker judges whether a sentence is supported by the document, which is what Ollama's page for it describes. A true
  sentence that the document never mentions can come back No.
• Code splits after a full stop, an exclamation mark or a question mark that is followed by a space and a capital letter. A
  title such as Dr. Lee therefore ends a piece. Its date guard lists German month names.
• There is one model call for each sentence, plus one for the summary. A long text takes a while on a small machine.
• Ollama's page says the document needs no splitting unless it passes 32K tokens.
• qwen2.5:1.5b is a small model. Pick a larger one in Ollama Model if its summary is poor.
• The model's answers are not checked for form. A reply other than No is dropped by Filter.
• Everything stays on the machine that runs Ollama, unless you point the credential somewhere else.