GLM-5.3-Flash EXL3 4bpw for TensorFold

GLM-5.3-Flash EXL3 4bpw for TensorFold

4-bit EXL3 quantization of GLM-5.3-Flash prepared for TensorFold, with lower KL divergence than a popular TR3-4bpw quant at the same size. For people serving large models on DGX Spark machines.

Made by Mia-AiLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 4-bit EXL3 version of zai-org/GLM-5.3-Flash made by Mia's AI Lab, prepared to run with TensorFold. It keeps the same format and bit width as a popular community quant called TR3-4bpw, and only the calibration is new. By the lab's measurements, its drift from the official model is lower on all six test sets, with size, speed and memory unchanged.

Who it suits: It suits people who serve GLM-5.3-Flash on their own hardware, especially two DGX Spark machines with TensorFold, and want the closest match to the official FP8 release.

Industry | Who uses it | How they use it | Example
Software | Self-hosting developers | Serve a large model on two DGX Spark machines | Running the TensorFold recipe for a coding assistant
Research | Quantization researchers | Compare calibration methods at equal bit width | Checking KL divergence against the FP8 reference
Platform | Inference engineers | Test long-context serving with a 1M-token window | Confirming that a long needle prompt is found

You can skip it if you do not run EXL3 or TensorFold, or if you need a smaller file, since it is about 176 GB.

Where to use it: The card tested it on TensorFold v0.6.0 with a recipe for two DGX Spark machines. The download works on Windows, macOS or Linux with Python.

• TensorFold, the serving stack it was built for
• Two DGX Spark machines, as in the tested recipe
• Windows, macOS or Linux for the download step

Use case | Who | What they get back
Local GLM serving | Hobbyists and labs | Decode speeds of 61.8 tokens per second with one request
Long context | Platform teams | A 1M-token window with a pool of 2,582,528 KV tokens
Coding and math | Developers | Scores of 98.8 percent on GSM8K and 95.7 percent on HumanEval

How to set it up: You need Python and about 176 GB of free disk. The card points to a separate recipe for serving on DGX Sparks with TensorFold and does not give run commands in the visible part.

• Python and the Hugging Face tool
• About 176 GB of disk
• TensorFold and the lab's recipe for serving

1. Install the Hugging Face download tool.
2. Download the checkpoint files to a large drive.
3. Follow the lab's TensorFold recipe to serve them.

pip install -U huggingface_hub
hf download Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold

How to use it: After serving it through TensorFold, send chat requests as you would to any GLM-5.3-Flash endpoint. The card reports tool calls work, with array arguments staying as JSON arrays, and drafted replies match serial ones.

Tool | What it does
hf download | Fetches the checkpoint files
TensorFold | Serves the checkpoint, here on two DGX Spark machines

Example: You serve the model and send a long coding request. The card reports about 78.6 tokens per second for code with one request, and a 1M-token test prompt where the needle was found.

For newcomers and experts: If you are new, this is a compressed copy of a large model meant for people who already run local serving. Read the lab's recipe before downloading, since the file is huge.

If you know the field, the routed experts are 4-bit EXL3 with an mcg codebook, and only calibration differs from TR3. KL to the FP8 release fell by 11 to 27 percent with experts only, and 2 to 18 percent as served, where TensorFold adds its own 4-bit dense weights.

GLM-5.3-Flash BF16 model
Routed experts quantized to 4-bit EXL3
New calibration lowers KL divergence
TensorFold serves it on DGX Sparks

Good to know: Coding and math score differences against TR3 are a handful of problems in both directions, so the lab points to KL as the real gain.

• License: Apache 2.0
• Cost to run: no API key, account or paid service is named, but you need hardware for a 176 GB model
• Data it reaches: files downloaded from Hugging Face, plus your own prompts on your machine
• Not for: small GPUs or laptops, and setups outside the tested TensorFold recipe

Title: KL divergence to the FP8 release on wiki, lower is better
TR3-4bpw: 0.0990
This quant: 0.0903

On Hugging Face, 13 November 2026

• Task: image text to text
• Parameters: 87.8B
• Weights: about 176 GB in I16
• Formats: Safetensors
• Downloads in the last 30 days: 4,746
• Likes: 73
• License: apache-2.0
• Based on: zai-org/GLM-5.3-Flash
• Quantized versions: 1
• Fine-tunes: 1
• Tags: safetensors, glm5_next, exl3, tensorfold, quantized, dgx-spark, image-text-to-text, conversational, 4-bit

Links

• Model: https://huggingface.co/Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold
• Files: https://huggingface.co/Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Mia-AiLab/GLM-5.3-Flash-EXL3-4bpw-TensorFold

Then follow the usage in its model card. Nothing in this kit runs by itself.