--- title: "ERNIE-4.5-21B-A3B-Thinking-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/30abbcf9-4b66-4ff1-bc03-3be6c5db61c2" updated: "2026-11-13" --- # ERNIE-4.5-21B-A3B-Thinking-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Works through logic, math, science and coding problems with a 21B mixture-of-experts model that activates 3B parameters per token, in GGUF files by Unsloth. Supports tool calls and 128K context. ERNIE-4.5-21B-A3B-Thinking-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is This repository holds GGUF versions of ERNIE-4.5-21B-A3B-Thinking, a text model from Baidu that reasons at length before it answers. It has 21B parameters in total and 3B active for each token. Unsloth made the quantized files, and the record lists English and Chinese. ### Who it suits Developers and researchers who want a mid-size reasoning model with tool use and long context on their own hardware. The card advises it for highly complex reasoning tasks, since this version thinks for longer. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a reasoning model with function calls in their own stack | Let it call a weather function from a chat | | Research | AI researchers | Test reasoning on maths, science and coding problems | Compare its thinking length on hard questions | | Education | Students | Study how a reasoning model works through a problem | Ask for a proof and read each step | | Self-hosting | Small teams | Keep a long-context model on their own machine | Feed it a long document of up to 131072 tokens | People who want quick, short replies to simple questions can skip it, because the longer thinking slows each answer. ### Where to use it - Windows, macOS or Linux with Python, for downloading the files - FastDeploy, the server the card shows, which is set up for the original weights - The card also expects vLLM and transformers to load Transformer-style weights | Use case | Who | What they get back | | --- | --- | --- | | Hard reasoning | Students and researchers | A long, worked answer to logic, maths or science questions | | Coding help | Developers | Reasoned code suggestions | | Tool use | Developers | Function calls the model chooses to make | ### How to set it up - Python, for the Hugging Face hub tools - Disk space for the file you choose, with the 4-bit weights estimated at about 10.9 GB - For the FastDeploy route on the original weights, one 80 GB GPU and FastDeploy version 2.2 1. Install the Hugging Face hub tools with pip. 2. Download the files with the hf command. 3. Pick a runner that can load the GGUF file, since the card shows no GGUF run command. ```sh pip install -U huggingface_hub hf download unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF ``` ### How to use it The card's quick start serves the original Baidu weights with FastDeploy, using the command below. It starts an OpenAI style server, with a reasoning parser, a tool call parser and the full 131072 token context. Function calling is supported. ```bash python -m fastdeploy.entrypoints.openai.api_server \ --model baidu/ERNIE-4.5-21B-A3B-Thinking \ --port 8180 \ --metrics-port 8181 \ --engine-worker-queue-port 8182 \ --load_choices "default_v1" \ --tensor-parallel-size 1 \ --max-model-len 131072 \ --reasoning-parser ernie_x1 \ --tool-call-parser ernie_x1 \ --max-num-seqs 32 ``` | Tool | What it does | | --- | --- | | --model | Names the model to serve | | --port | Sets the server port, 8180 here | | --max-model-len | Sets the context length, 131072 here | | --reasoning-parser | Splits the model's thinking from its answer | | --tool-call-parser | Reads the model's function calls | | --max-num-seqs | Caps how many sequences run at once, 32 here | ### Example You ask about the weather in Beijing today and give the model a get\_weather function to call. The card uses this setup to show function calling, where the model can answer with a call to that function. ### For newcomers and experts This is a reasoning model, so expect it to think for a while before it answers. A sensible first step is to download the files and pick a runner that reads GGUF. Start with a hard question, not a quick one. The model is a text mixture-of-experts with 28 layers and 20 query heads sharing 4 key value heads. Each token uses 6 of 64 text experts plus 2 shared experts. The card says this version has an increased thinking length, and it targets reasoning, tool use and 128K context. Send a question, with or without a tool list → The model thinks at length → A parser separates thinking from the answer → You get the answer or a function call ### Good to know - Thinking: This version has a longer thinking length, and the card recommends it for highly complex reasoning tasks. - Hardware: The FastDeploy route in the card needs one 80 GB GPU and FastDeploy version 2.2. - Context: The context length is 131072 tokens. - License: Apache 2.0. - Cost to run: none named. The FastDeploy route needs an 80 GB GPU, and no paid service appears in the source. - Data it reaches: Files on your computer and the model files from Hugging Face. - Not for: Quick replies to simple questions, since the card points this version at highly complex reasoning. ## README ````markdown # ERNIE-4.5-21B-A3B-Thinking-GGUF Works through logic, math, science and coding problems with a 21B mixture-of-experts model that activates 3B parameters per token, in GGUF files by Unsloth. Supports tool calls and 128K context. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF versions of ERNIE-4.5-21B-A3B-Thinking, a text model from Baidu that reasons at length before it answers. It has 21B parameters in total and 3B active for each token. Unsloth made the quantized files, and the record lists English and Chinese. Who it suits: Developers and researchers who want a mid-size reasoning model with tool use and long context on their own hardware. The card advises it for highly complex reasoning tasks, since this version thinks for longer. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a reasoning model with function calls in their own stack | Let it call a weather function from a chat Research | AI researchers | Test reasoning on maths, science and coding problems | Compare its thinking length on hard questions Education | Students | Study how a reasoning model works through a problem | Ask for a proof and read each step Self-hosting | Small teams | Keep a long-context model on their own machine | Feed it a long document of up to 131072 tokens ``` People who want quick, short replies to simple questions can skip it, because the longer thinking slows each answer. Where to use it: - Windows, macOS or Linux with Python, for downloading the files - FastDeploy, the server the card shows, which is set up for the original weights - The card also expects vLLM and transformers to load Transformer-style weights ```table Use case | Who | What they get back Hard reasoning | Students and researchers | A long, worked answer to logic, maths or science questions Coding help | Developers | Reasoned code suggestions Tool use | Developers | Function calls the model chooses to make ``` How to set it up: - Python, for the Hugging Face hub tools - Disk space for the file you choose, with the 4-bit weights estimated at about 10.9 GB - For the FastDeploy route on the original weights, one 80 GB GPU and FastDeploy version 2.2 1. Install the Hugging Face hub tools with pip. 2. Download the files with the hf command. 3. Pick a runner that can load the GGUF file, since the card shows no GGUF run command. ```sh pip install -U huggingface_hub hf download unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF ``` How to use it: The card's quick start serves the original Baidu weights with FastDeploy, using the command below. It starts an OpenAI style server, with a reasoning parser, a tool call parser and the full 131072 token context. Function calling is supported. ```bash python -m fastdeploy.entrypoints.openai.api_server \ --model baidu/ERNIE-4.5-21B-A3B-Thinking \ --port 8180 \ --metrics-port 8181 \ --engine-worker-queue-port 8182 \ --load_choices "default_v1" \ --tensor-parallel-size 1 \ --max-model-len 131072 \ --reasoning-parser ernie_x1 \ --tool-call-parser ernie_x1 \ --max-num-seqs 32 ``` ```table Tool | What it does --model | Names the model to serve --port | Sets the server port, 8180 here --max-model-len | Sets the context length, 131072 here --reasoning-parser | Splits the model's thinking from its answer --tool-call-parser | Reads the model's function calls --max-num-seqs | Caps how many sequences run at once, 32 here ``` Example: You ask about the weather in Beijing today and give the model a get_weather function to call. The card uses this setup to show function calling, where the model can answer with a call to that function. For newcomers and experts: This is a reasoning model, so expect it to think for a while before it answers. A sensible first step is to download the files and pick a runner that reads GGUF. Start with a hard question, not a quick one. The model is a text mixture-of-experts with 28 layers and 20 query heads sharing 4 key value heads. Each token uses 6 of 64 text experts plus 2 shared experts. The card says this version has an increased thinking length, and it targets reasoning, tool use and 128K context. ```flow Send a question, with or without a tool list The model thinks at length A parser separates thinking from the answer You get the answer or a function call ``` Good to know: - Thinking: This version has a longer thinking length, and the card recommends it for highly complex reasoning tasks. - Hardware: The FastDeploy route in the card needs one 80 GB GPU and FastDeploy version 2.2. - Context: The context length is 131072 tokens. - License: Apache 2.0. - Cost to run: none named. The FastDeploy route needs an 80 GB GPU, and no paid service appears in the source. - Data it reaches: Files on your computer and the model files from Hugging Face. - Not for: Quick replies to simple questions, since the card points this version at highly complex reasoning. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 21.8B - Weights: about 10.9 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 6,361 - Likes: 62 - License: apache-2.0 - Languages: en, zh - Based on: baidu/ERNIE-4.5-21B-A3B-Thinking - Tags: transformers, gguf, ernie4.5, text-generation, conversational ## Links - Model: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF - Files: https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF on Hugging Face](https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | ERNIE-4.5-21B-A3B-Thinking-GGUF (this page) | 6,361 | 62 | 21.8B | apache-2.0 | 2026-01-24 | | [ERNIE-4.5-21B-A3B-Thinking](https://mindket.com/p/d4df4086-9423-4162-90d1-c7c4b9ffe166) | 64,420 | 796 | 21.8B | apache-2.0 | 2025-11-26 | | [ERNIE-4.5-VL-28B-A3B-Thinking](https://mindket.com/p/0ece86e1-af9c-4fdf-af90-bf60c2c869f0) | 2,021 | 548 | 29.7B | apache-2.0 | 2026-03-06 | | [Ornith-1.5-9B-GGUF](https://mindket.com/p/5dd265f6-9a3e-450b-ad2a-b49a78e2700a) | 5,772,425 | 673 | 9.2B | mit | 2026-08-24 | | [Ornith-1.5-35B-A3B-GGUF](https://mindket.com/p/c98822f4-2255-450a-89ce-df15fc0003a1) | 4,164,876 | 603 | 35.5B | mit | 2026-08-24 | | [Ornith-1.0-9B-GGUF](https://mindket.com/p/80667786-2895-41cb-b1d8-f6f6ae12215d) | 2,357,551 | 702 | 9B | mit | 2026-06-25 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-01-24 - First published: 2025-09-10 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF on Hugging Face](https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/30abbcf9-4b66-4ff1-bc03-3be6c5db61c2) - [Its page on Mindket](https://mindket.com/p/30abbcf9-4b66-4ff1-bc03-3be6c5db61c2)