GLM-5.3-Flash EXL3 3.5bpw

GLM-5.3-Flash EXL3 3.5bpw

Quantized copy of GLM-5.3-Flash in EXL3 format, 156 GB, with narrower routed experts, about 11 percent smaller than the same lab's 4-bit version. For people running large models locally on GPUs.

Made by Mia-AiLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Mia's AI Lab compressed zai-org/GLM-5.3-Flash into the EXL3 format, keeping its text and image input. It is a smaller sibling of a 4-bit version from the same lab. Only the routed experts are narrower, and everything else stays unquantized. The checkpoint is 156.3 GB.

Who it suits: It suits people who run big models on local GPUs with exllamav3 and want to save about 19 GB compared with the 4-bit sibling.

Industry | Who uses it | How they use it | Example
Software | Local model users | Fit a large model onto smaller GPU setups | Picking the smaller of two quantized files
Research | Quantization researchers | Study how expert bit width affects fidelity | Comparing KL divergence against the BF16 model
Platform | Inference engineers | Test smaller expert formats for serving | Trying the 3-bit expert path

You can skip it if you need the highest fidelity, since the card says this version costs some accuracy against the 4-bit sibling.

Where to use it: The card measured it with exllamav3 on RTX PRO 6000 Blackwell GPUs. The download works on Windows, macOS or Linux with Python.

• exllamav3, which the author used for the measurements
• Large GPUs, such as the RTX PRO 6000 Blackwell used in the tests
• Not yet tested with TensorFold serving

Use case | Who | What they get back
Local serving of GLM-5.3-Flash | Hobbyists | A checkpoint 11 percent smaller than the 4-bit sibling
Fidelity study | Researchers | KL and top-1 agreement numbers against BF16
Mixed expert widths | Quantizers | Per-layer bit allocation for the experts

How to set it up: You need Python and a lot of disk, since the files total about 156 GB. The card lists the checkpoint parts and a configuration file that gives the width of each expert tensor. It names no further setup steps in the visible part.

• Python and the Hugging Face tool
• About 156 GB of disk
• A runtime that reads EXL3, such as exllamav3

1. Install the Hugging Face download tool.
2. Download the repository files to a drive with enough room.
3. Load the checkpoint in a runtime that supports EXL3.

pip install -U huggingface_hub
hf download Mia-AiLab/GLM-5.3-Flash-EXL3-3.5bpw

How to use it: Load it like the 4-bit sibling, in the same format and layout. It uses the chat template from Z.ai that ships with the files. Speed, memory use and task benchmark scores are not measured yet.

Tool | What it does
hf download | Fetches the checkpoint files
exllamav3 | Runtime the card used to measure the model

Example: You load the checkpoint and run a long coding session with thinking on. Its next-token choices match the full BF16 model's top choice 97.54 percent of the time, against 98.06 for the 4-bit sibling.

For newcomers and experts: If you are new, this is a shrunk copy of a large model that trades a little accuracy for a smaller file. Download only if you already have a runtime and GPUs for models of this size.

If you know the field, the routed experts average 3.50 bits. Down projections are 4-bit everywhere. In 32 of the 42 MoE layers, the gate and up matrices use 3 bits instead. KL to BF16 with thinking on is about 1.6 to 1.7 times the 4-bit sibling's, and 1.19 times with thinking off.

GLM-5.3-Flash BF16 weights
Routed experts quantized to EXL3
Gate and up projections at 3 bits in 32 layers
Everything else left unquantized
156 GB checkpoint

Good to know: All figures compare next-token behavior with the BF16 model and say nothing yet about task scores. In three small subsets the difference is within noise.

• License: Apache 2.0
• Cost to run: no API key, account or paid service is named, but you need GPUs with a lot of memory
• Data it reaches: files downloaded from Hugging Face, plus your own prompts on your machine
• Not for: uses that need the best fidelity, or setups the card marks as untested such as TensorFold serving

Title: Checkpoint size in GB
4-bit sibling: 175.6 GB
This 3.5 bit quant: 156.3 GB

On Hugging Face, 13 November 2026

• Task: image text to text
• Parameters: 78.1B
• Weights: about 156 GB in I16
• Formats: Safetensors
• Downloads in the last 30 days: 140
• Likes: 4
• License: apache-2.0
• Based on: zai-org/GLM-5.3-Flash
• Tags: safetensors, glm5_next, exl3, quantized, image-text-to-text, conversational

Links

• Model: https://huggingface.co/Mia-AiLab/GLM-5.3-Flash-EXL3-3.5bpw
• Files: https://huggingface.co/Mia-AiLab/GLM-5.3-Flash-EXL3-3.5bpw/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Mia-AiLab/GLM-5.3-Flash-EXL3-3.5bpw

Then follow the usage in its model card. Nothing in this kit runs by itself.