AREX-2-i1-GGUF

AREX-2-i1-GGUF

Weighted GGUF quantizations of BAAI/AREX-2, a 26.9B vision model, in about two dozen sizes from 7.2 GB to 22.2 GB, plus an imatrix file for making your own quants. Made by mradermacher.

Made by mradermacher on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds weighted GGUF quantizations of BAAI/AREX-2, a 26.9B model that the card calls a vision model. Quantizing shrinks the weights so the model fits on smaller hardware, at some cost in quality. The list has about two dozen sizes, from 7.2 GB to 22.2 GB, plus a 0.1 GB imatrix file for making your own. The card does not say what AREX-2 itself is for, so read the base model's page too.

Who it suits: It suits people who run models locally and need a version of AREX-2 sized to their memory, and tinkerers who compare quantization levels.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists with a strong GPU or lots of RAM | Run a 26.9B vision model locally in a smaller form | Pick the Q4_K_M file that the card recommends
Model research | People who compare quantization | Weigh quality against size across types | Compare an IQ file with a similar-sized Q file, as the card suggests
Model packaging | People who make their own quants | Start from the imatrix file | Use the 0.1 GB imatrix file to build a custom quant

If you want the original full-precision model, or the static quants, use their own repositories instead.

Where to use it:

• Local runners that load GGUF files
• Computers with room for the file you choose, from about 7.2 GB to 22.2 GB
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Local testing | Hobbyists | A GGUF file you load in your runner
Size comparison | Researchers | Many quant types of one model to compare
Custom quant | Packagers | An imatrix file to build your own

How to set it up: What you need first:

• Python and the Hugging Face command line
• A runner that can load GGUF files
• Free space for the file you pick, between 7.2 and 22.2 GB

1. Choose a quant type from the table below.
2. Download that one file from the repository's file list.
3. Load it in your GGUF runner.

pip install -U huggingface_hub
hf download mradermacher/AREX-2-i1-GGUF

That command fetches every file, so for a single quant pick the file from the file list on the model page instead.

How to use it: The card gives no run command. It points to TheBloke's READMEs for how to use GGUF files, including how to join files that come in several parts. Because this is a vision model, any mmproj files sit in the separate static repository.

Title: Selected quants
Type | Size in GB | Card's note
IQ1_S | 7.2 | A last resort for small memory
Q2_K_S | 10.3 | Poor quality, says the card
IQ3_M | 12.7 | None given
Q4_K_S | 15.7 | The card's best mix of size, speed and quality
Q4_K_M | 16.6 | Fast, and recommended by the card
Q6_K | 22.2 | Close to the static Q6_K

Part | What it does
imatrix file | A 0.1 GB file used to create your own quants
i1 quants | Weighted quants made with an imatrix
Static repository | Holds the static quants and any mmproj files for the vision side

Example: You pick the Q4_K_M file at 16.6 GB, which the card recommends, and load it in your GGUF runner. If you need the vision side, you also fetch the mmproj file from the static repository.

For newcomers and experts: A good first try is Q4_K_M, since the card recommends it. If your memory is short, step down a size. The card says IQ types are often a better pick than a non-IQ type of similar size.

For experts, the i1 files are weighted quants made with an importance matrix, and the list is sorted by size, not by quality. For some types the card names a better neighbor. It says IQ3_S beats the Q3_K types, and that IQ3_XS is probably better than Q3_K_S. The imatrix file is provided so you can make your own quants. The card also points to graphs that compare lower-quality quant types, and to some thoughts by Artefact2.

BAAI/AREX-2 is the original model
It is converted and quantized with an imatrix
You pick a size from the list
Your runner loads the GGUF file

Good to know:

Title: File size in GB
IQ1_S: 7.2 GB
IQ2_M: 10.1 GB
IQ3_M: 12.7 GB
Q4_K_S: 15.7 GB
Q4_K_M: 16.6 GB
Q5_K_M: 19.3 GB
Q6_K: 22.2 GB

• Order: the list is sorted by size, not by quality.
• Quality: the card warns that the smallest files and a few mid-size types give lower quality.
• Vision: this is a vision model, and any mmproj files sit in the static repository.
• Base model: BAAI/AREX-2, whose own page you should read for its intended use.
• Requests: the author takes requests for other models to quantize, through a page the card points to.
• License: apache-2.0
• Cost to run: no API key or paid service named. You need memory for the file you pick.
• Data it reaches: it downloads files from Hugging Face.
• Not for: expecting the quality of the original from the smallest files.

On Hugging Face, 13 November 2026

• Library: transformers
• Parameters: 26.9B
• Weights: about 13.4 GB at 4-bit (estimate)
• Formats: Safetensors, GGUF
• Downloads in the last 30 days: 2,207
• Likes: 11
• License: apache-2.0
• Languages: en
• Based on: BAAI/AREX-2
• Tags: transformers, gguf, qwen3_5, agent, deep-research, reasoning, tool-use, long-context, self-improvement, safetensors, imatrix, conversational

Links

• Model: https://huggingface.co/mradermacher/AREX-2-i1-GGUF
• Files: https://huggingface.co/mradermacher/AREX-2-i1-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download mradermacher/AREX-2-i1-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.