ERNIE-4.5-21B-A3B-Thinking-GGUF

ERNIE-4.5-21B-A3B-Thinking-GGUF

Works through logic, math, science and coding problems with a 21B mixture-of-experts model that activates 3B parameters per token, in GGUF files by Unsloth. Supports tool calls and 128K context.

Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF versions of ERNIE-4.5-21B-A3B-Thinking, a text model from Baidu that reasons at length before it answers. It has 21B parameters in total and 3B active for each token. Unsloth made the quantized files, and the record lists English and Chinese.

Who it suits: Developers and researchers who want a mid-size reasoning model with tool use and long context on their own hardware. The card advises it for highly complex reasoning tasks, since this version thinks for longer.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a reasoning model with function calls in their own stack | Let it call a weather function from a chat
Research | AI researchers | Test reasoning on maths, science and coding problems | Compare its thinking length on hard questions
Education | Students | Study how a reasoning model works through a problem | Ask for a proof and read each step
Self-hosting | Small teams | Keep a long-context model on their own machine | Feed it a long document of up to 131072 tokens

People who want quick, short replies to simple questions can skip it, because the longer thinking slows each answer.

Where to use it:

• Windows, macOS or Linux with Python, for downloading the files
• FastDeploy, the server the card shows, which is set up for the original weights
• The card also expects vLLM and transformers to load Transformer-style weights

Use case | Who | What they get back
Hard reasoning | Students and researchers | A long, worked answer to logic, maths or science questions
Coding help | Developers | Reasoned code suggestions
Tool use | Developers | Function calls the model chooses to make

How to set it up:

• Python, for the Hugging Face hub tools
• Disk space for the file you choose, with the 4-bit weights estimated at about 10.9 GB
• For the FastDeploy route on the original weights, one 80 GB GPU and FastDeploy version 2.2

1. Install the Hugging Face hub tools with pip.
2. Download the files with the hf command.
3. Pick a runner that can load the GGUF file, since the card shows no GGUF run command.

pip install -U huggingface_hub
hf download unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF

How to use it: The card's quick start serves the original Baidu weights with FastDeploy, using the command below. It starts an OpenAI style server, with a reasoning parser, a tool call parser and the full 131072 token context. Function calling is supported.

python -m fastdeploy.entrypoints.openai.api_server \
       --model baidu/ERNIE-4.5-21B-A3B-Thinking \
       --port 8180 \
       --metrics-port 8181 \
       --engine-worker-queue-port 8182 \
       --load_choices "default_v1" \
       --tensor-parallel-size 1 \
       --max-model-len 131072 \
       --reasoning-parser ernie_x1 \
       --tool-call-parser ernie_x1 \
       --max-num-seqs 32

Tool | What it does
--model | Names the model to serve
--port | Sets the server port, 8180 here
--max-model-len | Sets the context length, 131072 here
--reasoning-parser | Splits the model's thinking from its answer
--tool-call-parser | Reads the model's function calls
--max-num-seqs | Caps how many sequences run at once, 32 here

Example: You ask about the weather in Beijing today and give the model a get_weather function to call. The card uses this setup to show function calling, where the model can answer with a call to that function.

For newcomers and experts: This is a reasoning model, so expect it to think for a while before it answers. A sensible first step is to download the files and pick a runner that reads GGUF. Start with a hard question, not a quick one.

The model is a text mixture-of-experts with 28 layers and 20 query heads sharing 4 key value heads. Each token uses 6 of 64 text experts plus 2 shared experts. The card says this version has an increased thinking length, and it targets reasoning, tool use and 128K context.

Send a question, with or without a tool list
The model thinks at length
A parser separates thinking from the answer
You get the answer or a function call

Good to know:

• Thinking: This version has a longer thinking length, and the card recommends it for highly complex reasoning tasks.
• Hardware: The FastDeploy route in the card needs one 80 GB GPU and FastDeploy version 2.2.
• Context: The context length is 131072 tokens.
• License: Apache 2.0.
• Cost to run: none named. The FastDeploy route needs an 80 GB GPU, and no paid service appears in the source.
• Data it reaches: Files on your computer and the model files from Hugging Face.
• Not for: Quick replies to simple questions, since the card points this version at highly complex reasoning.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 21.8B
• Weights: about 10.9 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 6,361
• Likes: 62
• License: apache-2.0
• Languages: en, zh
• Based on: baidu/ERNIE-4.5-21B-A3B-Thinking
• Tags: transformers, gguf, ernie4.5, text-generation, conversational

Links

• Model: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF
• Files: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.