Falcon OCR

Falcon OCR

Reads a document image with a small 270M-parameter model and returns the text, formulas as LaTeX or tables as HTML, including scans, receipts and handwriting.

Made by tiiuae on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Falcon OCR reads a picture of a document with a single 270M-parameter model that mixes image patches and text from the first layer. Given an image, it writes out ordinary text, formulas as LaTeX, or tables as HTML.

Most OCR systems chain a vision encoder to a separate text decoder. Falcon OCR uses one Transformer for both, and you switch tasks through a category setting rather than extra modules.

Who it suits: Developers who need document text, tables or formulas out of images, and who would rather call one small model than assemble a pipeline.

Industry | Who uses it | How they use it | Example
Finance and admin | Back-office teams | Turn receipts and forms into text | Run a scanned receipt through the plain category
Archives | Librarians and archivists | Digitize historical and scanned pages | Read a page from a book scan
Science and education | Researchers and teachers | Pull formulas out of papers and exam sheets as LaTeX | Use the formula category on a photographed equation
Software | Developers | Add OCR with one model call | Get a table back as HTML for a data pipeline

People who want a finished desktop OCR app can skip it, since this is a Python model you call from code.

Where to use it:

• Python scripts with transformers and PyTorch 2.5 or newer
• A vLLM server in Docker, using the image the card links
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Read a scanned page | Developers | The page text as one string
Extract a table | Data teams | The table as HTML
Extract a formula | Researchers | The formula as LaTeX

How to set it up: You need Python and PyTorch 2.5 or newer, because the model uses FlexAttention.

• Python with pip
• PyTorch 2.5 or newer
• An image file of the document you want to read

1. Install the packages with pip.
2. Load the model with transformers. The card sets trust_remote_code to True, which runs the repository's own code.
3. Open your image with Pillow and call generate.

pip install "torch>=2.5" transformers pillow einops

The first call can be slower, because torch.compile builds optimized kernels. To fetch the files alone, Hugging Face gives these two commands.

pip install -U huggingface_hub
hf download tiiuae/Falcon-OCR

How to use it: Load the model, open an image and call generate. Without a category it reads the page as plain output, and you can pick a category to choose the format.

import torch
from PIL import Image
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "tiiuae/Falcon-OCR",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
image = Image.open("document.png")
texts = model.generate(image)  # default category is "plain"
print(texts[0])

texts = model.generate(image, category="text")     # plain text
texts = model.generate(image, category="formula")  # LaTeX
texts = model.generate(image, category="table")    # HTML table

Further category names cover page parts such as headers, footers, titles, captions, footnotes and list items. The function returns a list with one string per image.

Tool | What it does
model.generate | Reads one image or a list of images and returns one string per image
category="text" | Returns the text as plain text
category="formula" | Returns a formula as LaTeX
category="table" | Returns a table as HTML

Example: You open a photographed invoice with Image.open("document.png") and call model.generate(image, category="table"). You get back a list with one string, and that string holds the invoice table written as HTML.

For newcomers and experts: Think of it as a reader for pictures of pages. You hand it an image, tell it whether you want text, a formula or a table, and it gives you the result as text. A good first try is the short script above on one clear scan.

Underneath, image patches and text tokens share one parameter space. Image tokens attend in both directions, and text tokens decode causally with the image in view. Version 1.5 was post-trained with GRPO, using edit distance for text, LaTeX validity for formulas and TEDS for tables, and zero reward for loops, omissions and over-long output.

Open the document image
Pick a category such as table
Call model.generate
Read one output string per image

Good to know: The 1.5 update widened coverage to historical scans, handwriting, receipts, magazines, book scans, exam papers and hard tables, and aimed to cut invented text and repeated phrases.

• Speed: the authors report often 2 to 3 times higher throughput than 0.9B-class OCR models in their own vLLM setup, depending on sequence length and batch size
• Size: 270M parameters in Safetensors, about 1.1 GB in F32 and about 135 MB at 4-bit as an estimate
• Variants: Hugging Face lists 2 quantized and 5 fine-tuned versions made by other people
• License: Apache 2.0
• Cost to run: no API key or paid service is named
• Data it reaches: the images you pass in, which stay on your machine, plus the model files you download from Hugging Face
• Not for: the card lists no restrictions beyond its license

On Hugging Face, 13 November 2026

• Task: image to text
• Library: transformers
• Parameters: 270M
• Weights: about 1.1 GB in F32; about 135 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,667
• Likes: 178
• License: apache-2.0
• Quantized versions: 2
• Fine-tunes: 5
• Tags: transformers, safetensors, falcon_ocr, text-generation, falcon, vision-language, document-understanding, v1.5, image-to-text

Links

• Model: https://huggingface.co/tiiuae/Falcon-OCR
• Files: https://huggingface.co/tiiuae/Falcon-OCR/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download tiiuae/Falcon-OCR

Then follow the usage in its model card. Nothing in this kit runs by itself.