GLM-5.3-Flash EXL3 Spark

GLM-5.3-Flash EXL3 Spark

Runs GLM-5.3-Flash with a 256k-token context and image input on one DGX Spark, using a mixed 2.31 bits-per-weight EXL3 build. For people hosting a large model on a single compact machine who value quality over speed.

Made by 0xSero on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: GLM-5.3-Flash, squeezed into a mixed-precision EXL3 form that lets one NVIDIA DGX Spark hold a context of 256k tokens plus image input. The Spark has 128 GB of unified memory. Some expert layers get more bits for accuracy and the rest get fewer for size. The average is about 2.31 bits per weight, and the files take 96.1 GB.

Who it suits: People who host a large model on one DGX Spark or a similar machine and want the best accuracy a file this small can give.

Industry | Who uses it | How they use it | Example
Software | Local AI engineers | Host a large long-context model on one compact machine | A private assistant reading a long document
Research | Quantization researchers | Study how spending extra bits on chosen layers affects agreement | Compare the 12 upgraded layers with the base
Document work | Analysts | Ask questions over a huge context and attached images | A report plus its figures in one prompt
AI hobbyists | Spark owners | Run a big model at home | A chat on a single DGX Spark

You can skip it if you need fast output, since the card calls this a quality build, not a speed one.

Where to use it: It targets one NVIDIA DGX Spark, which has a GB10 chip and an arm64 processor, and it needs a runtime that reads EXL3 files.

• A DGX Spark with 128 GB of unified memory
• A runtime that supports EXL3 files, as the card's serving section describes
• The Hugging Face tool, to download the 12 shards

Use case | Who | What they get back
Long-context chat | Developers | Answers over prompts up to 262,144 tokens
Image questions | Analysts | Replies that take in pictures, at about 8,000 tokens each
Quality testing | Researchers | Measured agreement with the teacher on 32 rows

How to set it up: You need a DGX Spark or similar machine, about 96.1 GB of disk space and a runtime for EXL3 files. The card has a serving section with its validated setup.

• A DGX Spark or a similar machine with plenty of unified memory
• About 96.1 GB of disk space for 12 shards
• A runtime that loads EXL3 files

1. Install the Hugging Face download tool.
2. Download the 12 shards to your machine.
3. Follow the serving section of the card for the context length and settings it validated.

pip install -U huggingface_hub
hf download 0xSero/GLM-5.3-Flash-EXL3-Spark

How to use it: Serve the model with a context length up to 262,144 tokens, then chat or send long documents and images. The card says it currently runs without speculative decoding, which roughly halves decode speed against EXL3 builds that can use it.

Tool | What it does
EXL3 runtime | Loads and serves the quantized weights
max_model_len | Sets the context length, validated at 262,144
Hugging Face tool | Downloads the 12 shards

Example: Send a prompt as long as a report of close to 236,510 tokens, which the card served end to end on one Spark. Each attached image costs about 8,000 image tokens, so budget your context for them.

For newcomers and experts: Quantizing stores a model's numbers with fewer bits so it fits on smaller hardware. This build gives a few chosen layers more bits than the rest, a mix that costs a little more size and gains accuracy. A first small try is to download it and run a short chat before testing long prompts.

For experts, the upgraded layers sit at positions 3, 32, 33 and 36 to 44, and each keeps all 288 experts at the higher 3.05 bits. The rest keep the base's 2.05 bits, and the attention and vision parts are unchanged. The card derives the 2.31 average two ways, from bytes and from layer mass. The trade is speed against accuracy, because multi-token prediction is not yet used.

Start from the 2.05 bpw GLM-5.3-Flash EXL3 base
Upgrade 12 expert layers to 3.05 bpw
Keep the other 33 layers at 2.05 bpw
Serve on one DGX Spark at 262,144 tokens
Read the measured quality against the base

Good to know: The numbers come from the card's own measurements on 65,504 positions across 32 rows, and the 2.05 bpw base was re-measured the same day.

• Quality: All 32 rows improve on top-1 agreement, KL and NLL, and none regress.
• Speed: One Spark gives about 10.79 tokens per second decoding and 503.5 prefill.
• No speculative decoding: The card traces this to two code blockers, not a weight problem.

Title: Top-1 agreement with the teacher
Base at 2.05 bpw: 0.78902
This build: 0.80842

Title: Candidate perplexity, lower is better
Base at 2.05 bpw: 4.28692 PPL
This build: 4.03563 PPL

• License: MIT, carried from the base model.
• Cost to run: none named, the source lists no API key, account or paid service, though it needs a machine with about 96 GB of space and large memory.
• Data it reaches: the prompts and images you send to your own server, plus the files you download.
• Not for: fast decoding, since the card says this build is about quality and runs without speculative decoding for now.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 48B
• Weights: about 95.9 GB in I16
• Formats: Safetensors
• Downloads in the last 30 days: 399
• Likes: 11
• License: mit
• Languages: en, zh
• Based on: turboderp/GLM-5.3-Flash-exl3
• Quantized versions: 1
• Tags: transformers, safetensors, glm5_next, image-text-to-text, exl3, quantized, dgx-spark, mixed-precision, long-context, text-generation, conversational

Links

• Model: https://huggingface.co/0xSero/GLM-5.3-Flash-EXL3-Spark
• Files: https://huggingface.co/0xSero/GLM-5.3-Flash-EXL3-Spark/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download 0xSero/GLM-5.3-Flash-EXL3-Spark

Then follow the usage in its model card. Nothing in this kit runs by itself.