AREX-2 GGUF

AREX-2 GGUF

GGUF quantized files of BAAI's AREX-2, a 26.9B model, from 2-bit to 8-bit with an optional multi-modal supplement. They suit people who run a large model locally and want to pick a size that fits their memory.

Made by mradermacher on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF quantized versions of AREX-2, a 26.9 billion parameter model from BAAI. The files were made by mradermacher, and they come in many sizes so you can trade quality for memory. The card does not describe what AREX-2 itself is for, so read BAAI's own model page before you rely on it. Two small supplement files are marked as multi-modal, which suggests image input is possible.

Who it suits: It suits people who run models on their own machine and need a file that fits their memory. It also suits developers who want to try AREX-2 without the full-size weights.

Title: Who might use it
Industry | Who uses it | How they use it | Example
Local AI | Hobbyists | Run a large model on a home computer | Pick a mid-size build that fits their memory
Software | Developers | Test the model in a GGUF-based tool | Compare two quantization levels on the same prompts
Research | Model evaluators | Check how quality changes with size | Compare the 2-bit and 8-bit files

If you want the original full-precision weights, you can skip these files and go to the base model.

Where to use it: GGUF files load in tools that support the GGUF format, such as llama.cpp based apps. The card does not name a specific tool, so check that yours handles this model.

• A GGUF-capable runner on Windows, macOS or Linux
• Python with pip, to download the files with the Hugging Face tools
• A machine with memory for the build you choose

Title: Jobs it does
Use case | Who | What they get back
Local text generation | Hobbyists | Output from a large model on their own hardware
Size comparison | Developers | A way to see the quality and memory trade-off
Multi-modal tests | Experimenters | Image input, if the supplement file works with your tool

How to set it up: You need Python with pip to fetch the files, and enough memory for the build you pick. The card's table lists sizes from 10.8 GB to 28.7 GB for the main files. It does not give run commands, so follow your runner's own guide for GGUF files.

• Python with pip
• A GGUF-capable runner
• Memory for the build you choose, from 10.8 GB to 28.7 GB

1. Install the Hugging Face download tool.
2. Download the repository, or only the file you want.
3. Load the file in your runner.

pip install -U huggingface_hub
hf download mradermacher/AREX-2-GGUF

How to use it: Choose a build from the card's table and load that file in your runner. The card marks the 4-bit builds Q4_K_S and Q4_K_M as fast and recommended, and Q8_0 as the best quality. Multi-part files may need joining, and the card points to another author's guide for that. The multi-modal supplement files are separate and load alongside the main file in tools that support them.

Title: Builds and sizes
Build | Size in GB | Notes
Q2_K | 10.8 | Smallest main file
Q3_K_M | 13.4 | Lower quality
Q4_K_S | 15.7 | Fast, recommended
Q4_K_M | 16.6 | Fast, recommended
Q6_K | 22.2 | Higher quality
Q8_0 | 28.7 | Fast, best quality
mmproj-Q8_0 | 0.7 | Multi-modal supplement

Example: You have a machine with about 24 GB of memory free, so you pick Q4_K_M at 16.6 GB. You download that one file and load it in your GGUF runner. You test a few prompts, then try Q6_K if you want more quality and have room.

For newcomers and experts: If you are new, quantizing shrinks a model so it fits in less memory, at some cost in quality. Start with Q4_K_M, since the card recommends it, and move up or down by what your machine can hold.

If you know the field, these are static quants. The card says weighted imatrix quants of the same model are in a separate repository named mradermacher/AREX-2-i1-GGUF, and that IQ quants are often preferable to non-IQ quants of similar size. The IQ4_XS file here is 15.3 GB.

You choose a build by memory
You download the GGUF file
Your runner loads it
You prompt the model locally

Good to know: Bigger files keep more quality and need more memory. The card sorts the table by size and says that size is not always the same as quality.

Title: File size by build, GB
Q2_K: 10.8
Q3_K_M: 13.4
Q4_K_M: 16.6
Q5_K_M: 19.3
Q6_K: 22.2
Q8_0: 28.7

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the model files from Hugging Face, and the prompts you give it
• Not for: anyone who needs full-precision weights, or anyone who needs the card to explain what AREX-2 does, which it does not

On Hugging Face, 13 November 2026

• Library: transformers
• Parameters: 26.9B
• Weights: about 13.4 GB at 4-bit (estimate)
• Formats: Safetensors, GGUF
• Downloads in the last 30 days: 8,118
• Likes: 20
• License: apache-2.0
• Languages: en
• Based on: BAAI/AREX-2
• Tags: transformers, gguf, qwen3_5, agent, deep-research, reasoning, tool-use, long-context, self-improvement, safetensors, conversational

Links

• Model: https://huggingface.co/mradermacher/AREX-2-GGUF
• Files: https://huggingface.co/mradermacher/AREX-2-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download mradermacher/AREX-2-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.