Unlimited-OCR-2-GGUF

Unlimited-OCR-2-GGUF

GGUF files of the Unlimited-OCR-2 document OCR model in BF16, Q8_0 and Q4_K_M, with a vision projector. They run only in a llama.cpp build with the repository's patch, for people reading scanned and multi-page documents locally.

Made by RX5950XT on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a GGUF version of baidu/Unlimited-OCR-2, an OCR model that reads document pages from images. It comes in BF16, Q8_0 and Q4_K_M, with a vision projector file and a multimodal imatrix. It only runs in a llama.cpp build with the patch included in the repository.

Who it suits: People who need to read scanned pages or multi-page documents on their own machine, and who are comfortable compiling a patched llama.cpp. The card says Chinese and English pages were used in its tests.

Industry | Who uses it | How they use it | Example
Document work | Back-office teams | Read scanned pages into text with layout tags | Parse a single scanned page
Legal and finance | Records teams | Process multi-page documents locally | Parse a three-page report
Software | OCR developers | Run a local document parser through the llama.cpp tools | Call the command-line tool from a script
Research | Quantization researchers | Study how quants affect OCR accuracy | Compare Q8_0 and Q4_K_M with BF16

Anyone who wants to load it in LM Studio or Ollama can skip it, because the card says those tools cannot load these files.

Where to use it:

• A llama.cpp build with the repository's patch applied, using its llama-mtmd-cli and llama-server tools
• Windows, through a small prebuilt overlay that goes on top of the official llama.cpp Windows package
• Linux and macOS, by building the patched llama.cpp yourself

Use case | Who | What they get back
Single page parsing | Document teams | Recognized text with layout tags and box coordinates
Multi-page parsing | Records teams | Text for each page of a document in one run
Custom quantization | Advanced users | Your own quants, using the multimodal imatrix file

How to set it up:

• Python with pip, to download the files
• git, cmake and a C++ compiler, which is Visual Studio 2022 with C++ tools on Windows, the standard build tools package on Linux or the Xcode command line tools on macOS
• The CUDA Toolkit, if you want to use an NVIDIA GPU
• The official llama.cpp source at release b11913, which the card tested

1. Install the Hugging Face hub package and download the files.
2. Download the official llama.cpp source and check out release b11913.
3. Apply the patch file from this repository, then build the two tools.

pip install -U huggingface_hub
hf download RX5950XT/Unlimited-OCR-2-GGUF

hf download RX5950XT/Unlimited-OCR-2-GGUF llama.cpp-patch/unlimited-ocr2.patch --local-dir .

git apply /path/to/unlimited-ocr2.patch
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-mtmd-cli llama-server

The card says to drop the CUDA flag if you have no NVIDIA GPU, and that macOS uses Metal automatically.

How to use it: Run llama-mtmd-cli with the language model file, the vision projector file and your page image. The card says every precision needs the projector file. This single-page command matches the original model's default mode.

./build/bin/llama-mtmd-cli \
  --jinja --temp 0 -c 32768 -n 8000 \
  --image page.png \
  -p "<PAGE><__media__>Multi page parsing."

Put your own value in its key setting first.

Tool | What it does
llama-mtmd-cli | Runs the model on one or more images from the command line
llama-server | Serves the model from a local server
git apply | Adds the Unlimited-OCR-2 support to llama.cpp
hf download | Fetches the model files or a single file such as the patch

Example: You run the command with Q8_0 and a scanned page named page.png. The model prints the recognized text with layout tags and box coordinates. For several pages, the card gives a second command with a larger context and a list of page images.

For newcomers and experts: If you are new, expect some setup work. Download the files, build the patched llama.cpp, then try one page with Q8_0, which the card recommends. Check the text against the page.

Experts will note why the patch exists. The decoder matches version 1 but the attention differs, using 2D page position encoding and sparse routed attention. Converting it like version 1 ran, but the card measured only 5.6 percent text match on a page. The patch implements the routing and keeps the router weights unquantized.

Title: Mean text similarity to BF16, 64 test pages
Q8_0: 99.46 percent
Q4_K_M: 99.18 percent

Title: File sizes
File | Size
BF16 | 5.48 GiB
Q8_0 | 2.92 GiB
Q4_K_M | 2.01 GiB
Vision projector F16 | 787 MiB

Download the GGUF and projector files
Apply the patch and build llama.cpp
Run llama-mtmd-cli on page images
Read the recognized text and tags

Good to know:

• Compatibility: mainline llama.cpp, LM Studio and Ollama cannot load these files, since the architecture is new
• Quality: the median text match for both quants is 100 percent, but the worst Q4_K_M page scored 79.39 percent
• Q4_K_M recipe: key layers stay at Q8_0 and the router stays unquantized, for a larger file than plain Q4_K_M
• License: MIT
• Cost to run: none named, no API key or account
• Data it reaches: it downloads files from Hugging Face and reads the page images you give it
• Not for: loading in tools that use mainline llama.cpp, or converting the model with the version 1 method, which gives wrong output

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 2.9B
• Weights: about 1.5 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 8,452
• Likes: 8
• License: mit
• Based on: baidu/Unlimited-OCR-2
• Tags: gguf, llama.cpp, document-parsing, vision-language, unlimited-ocr, quantized, imatrix, image-text-to-text, multilingual, conversational

Links

• Model: https://huggingface.co/RX5950XT/Unlimited-OCR-2-GGUF
• Files: https://huggingface.co/RX5950XT/Unlimited-OCR-2-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download RX5950XT/Unlimited-OCR-2-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.