--- title: "GLM-5.3-Flash EXL3 3.05bpw" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/783914ac-53d0-4c6b-adb4-69db026aa354" updated: "2026-11-13" --- # GLM-5.3-Flash EXL3 3.05bpw > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs GLM-5.3-Flash locally on two RTX 3090 cards with 128 GB of RAM and a 262,144-token context, using a verified 3.05 bpw EXL3 quantization. For people hosting a large model at home. GLM-5.3-Flash EXL3 3.05bpw is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by albucino ## About ### What it is A compressed copy of the GLM-5.3-Flash text generation model, set up to run on a home machine with two 24 GB graphics cards while keeping the full 262,144-token context. The repository holds the model card, a snapshot of the runtime and a lock file of checksums. The weights are not stored here. They come unchanged from turboderp's 3.05 bits-per-weight EXL3 release, and the runtime pulls them down and checks them. ### Who it suits People who want a large long-context model on their own hardware instead of a hosted service, and who already own two RTX 3090s with 128 GB of RAM. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers building agents | Point an agent client at the local OpenAI-compatible server | A coding agent that reads a whole project in one prompt | | Research | Long-document analysts | Send huge inputs and look for facts buried deep inside them | Retrieve a fact hidden near the 260K-token mark | | Privacy-minded teams | Small teams with a GPU workstation | Keep prompts on a machine they own | Draft internal text without a hosted service | | Hardware testing | Builders of local AI rigs | Measure what two consumer GPUs can do | Compare first-token time at 131K and 260K tokens | You can skip it if you lack two RTX 3090s and 128 GB of RAM, since the setup was validated on one such machine only. ### Where to use it It runs on a Linux machine with Docker and the NVIDIA Container Toolkit. Any client that speaks the OpenAI API format can talk to the local server it starts. - Linux with Docker and the NVIDIA Container Toolkit - Two RTX 3090 GPUs and 128 GB of RAM - Agent tools and apps that can call an OpenAI-compatible API on your own machine - Windows, macOS or Linux with Python, only to fetch this repository's files | Use case | Who | What they get back | | --- | --- | --- | | Long-document questions | Researchers | Answers over prompts up to 262,144 tokens | | Agent work | Developers | Tool-calling turns, with reasoning limited to 4,096 tokens and prefix caching on | | Local serving | Hobbyists | An API on their own machine that decodes at 49.4 tokens per second in the card's test | ### How to set it up Prepare the hardware first, then fetch the files, then build and start the runtime. The weights are large, so leave room on disk. - Linux, Docker and the NVIDIA Container Toolkit - Two RTX 3090s and 128 GB of RAM - Python, for the Hugging Face download commands - About 117 GiB of disk space for the verified weights 1. Install the Hugging Face tools and fetch this repository's files. 2. Get the runtime project named glm53-flash-2x3090 by DominikBucko from GitHub. 3. Run its download script to fetch and verify the weights. 4. Copy the example environment file and set MODEL\_DIR in it. 5. Build the image, run the preflight check, then start the server. ```bash pip install -U huggingface_hub hf download albucino/GLM-5.3-Flash-EXL3-3.05bpw ``` ```bash scripts/download_model.sh /models/glm53-flash-exl3-3.05bpw cp .env.example .env make build-image && make preflight && make serve ``` ### How to use it Start the server with make serve. It exposes an OpenAI-compatible API on the local machine, so you point your client or agent tool at it and send prompts as usual. The card describes two profiles, one tuned for long prefill and one for agent work with prefix caching. | Tool | What it does | | --- | --- | | scripts/download\_model.sh | Downloads the weights and verifies them | | scripts/install\_model.py | Adds the 22 missing indexer entries and checks the files against the lock | | make build-image | Builds the Docker image | | make preflight | Runs the preflight check before you serve | | make serve | Starts the OpenAI-compatible API | ### Example Send the server a prompt of 131,072 tokens, such as a large document, in the prefill profile. The card's own run saw the first token after 113.5 seconds, then text at 49.4 tokens per second. ### For newcomers and experts GLM-5.3-Flash is a large language model, and quantizing it stores its numbers in fewer bits so it fits on home hardware. This page is the recipe, not the weights. A first small try is to run the two download commands, which fetch this repository's files before you commit to the large weights download. For experts, the routed experts sit in host memory while each GPU keeps an LRU cache of them in VRAM, with the hottest 20% of each layer pinned. For cold experts the GPU pulls 30% over PCIe and a CPU thread pool computes the rest, inside one CUDA graph per decode step. The upstream 3.05 bpw branch leaves 22 sparse-attention indexer tensors out of its index file, so a loader that trusts the index loses long-context retrieval. The install script adds them and checks all 29 files against the lock. The trade-offs are single-request serving, no speculative decoding and some quality lost against the BF16 original, with no size given. Fetch the 3.05 bpw EXL3 weights → Add 22 indexer entries to the index → Check all 29 files against the lock → Build the image and run preflight → Serve an OpenAI-compatible API ### Good to know The timings below come from the card's benchmark on one machine, one request at a time. Seconds to the first token | Label | Value | | --- | --- | | 131,072 prefill profile | 113.5 s | | 260,096 prefill profile | 235.3 s | | 131,072 agent profile | 236.2 s | | 8,192 agent profile | 9.5 s | - Hardware: validated on one machine only, with two RTX 3090s, a Threadripper PRO 5975WX and 128 GB of DDR4-3200 memory. - Speed: decode depends heavily on CPU memory bandwidth, and prefill depends on PCIe bandwidth. - Real agent turns: decoded at 37.7 tokens per second on average. - Cache: across 15 agent tasks, prefix caching served 72.9% of the prompt tokens. - License: MIT for GLM-5.3-Flash and for turboderp's quantization, Apache-2.0 for the runtime. Review the upstream terms before use. - Cost to run: none named, no API key, account or paid service, apart from the hardware. - Data it reaches: it downloads the weights from Hugging Face and serves an API on your own machine. - Not for: several requests at once, speculative decoding, or machines without two 24 GB GPUs. ## README ````markdown # GLM-5.3-Flash EXL3 3.05bpw Runs GLM-5.3-Flash locally on two RTX 3090 cards with 128 GB of RAM and a 262,144-token context, using a verified 3.05 bpw EXL3 quantization. For people hosting a large model at home. Made by albucino on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: A compressed copy of the GLM-5.3-Flash text generation model, set up to run on a home machine with two 24 GB graphics cards while keeping the full 262,144-token context. The repository holds the model card, a snapshot of the runtime and a lock file of checksums. The weights are not stored here. They come unchanged from turboderp's 3.05 bits-per-weight EXL3 release, and the runtime pulls them down and checks them. Who it suits: People who want a large long-context model on their own hardware instead of a hosted service, and who already own two RTX 3090s with 128 GB of RAM. ```table Industry | Who uses it | How they use it | Example Software | Developers building agents | Point an agent client at the local OpenAI-compatible server | A coding agent that reads a whole project in one prompt Research | Long-document analysts | Send huge inputs and look for facts buried deep inside them | Retrieve a fact hidden near the 260K-token mark Privacy-minded teams | Small teams with a GPU workstation | Keep prompts on a machine they own | Draft internal text without a hosted service Hardware testing | Builders of local AI rigs | Measure what two consumer GPUs can do | Compare first-token time at 131K and 260K tokens ``` You can skip it if you lack two RTX 3090s and 128 GB of RAM, since the setup was validated on one such machine only. Where to use it: It runs on a Linux machine with Docker and the NVIDIA Container Toolkit. Any client that speaks the OpenAI API format can talk to the local server it starts. - Linux with Docker and the NVIDIA Container Toolkit - Two RTX 3090 GPUs and 128 GB of RAM - Agent tools and apps that can call an OpenAI-compatible API on your own machine - Windows, macOS or Linux with Python, only to fetch this repository's files ```table Use case | Who | What they get back Long-document questions | Researchers | Answers over prompts up to 262,144 tokens Agent work | Developers | Tool-calling turns, with reasoning limited to 4,096 tokens and prefix caching on Local serving | Hobbyists | An API on their own machine that decodes at 49.4 tokens per second in the card's test ``` How to set it up: Prepare the hardware first, then fetch the files, then build and start the runtime. The weights are large, so leave room on disk. - Linux, Docker and the NVIDIA Container Toolkit - Two RTX 3090s and 128 GB of RAM - Python, for the Hugging Face download commands - About 117 GiB of disk space for the verified weights 1. Install the Hugging Face tools and fetch this repository's files. 2. Get the runtime project named glm53-flash-2x3090 by DominikBucko from GitHub. 3. Run its download script to fetch and verify the weights. 4. Copy the example environment file and set MODEL_DIR in it. 5. Build the image, run the preflight check, then start the server. ```bash pip install -U huggingface_hub hf download albucino/GLM-5.3-Flash-EXL3-3.05bpw ``` ```bash scripts/download_model.sh /models/glm53-flash-exl3-3.05bpw cp .env.example .env make build-image && make preflight && make serve ``` How to use it: Start the server with make serve. It exposes an OpenAI-compatible API on the local machine, so you point your client or agent tool at it and send prompts as usual. The card describes two profiles, one tuned for long prefill and one for agent work with prefix caching. ```table Tool | What it does scripts/download_model.sh | Downloads the weights and verifies them scripts/install_model.py | Adds the 22 missing indexer entries and checks the files against the lock make build-image | Builds the Docker image make preflight | Runs the preflight check before you serve make serve | Starts the OpenAI-compatible API ``` Example: Send the server a prompt of 131,072 tokens, such as a large document, in the prefill profile. The card's own run saw the first token after 113.5 seconds, then text at 49.4 tokens per second. For newcomers and experts: GLM-5.3-Flash is a large language model, and quantizing it stores its numbers in fewer bits so it fits on home hardware. This page is the recipe, not the weights. A first small try is to run the two download commands, which fetch this repository's files before you commit to the large weights download. For experts, the routed experts sit in host memory while each GPU keeps an LRU cache of them in VRAM, with the hottest 20% of each layer pinned. For cold experts the GPU pulls 30% over PCIe and a CPU thread pool computes the rest, inside one CUDA graph per decode step. The upstream 3.05 bpw branch leaves 22 sparse-attention indexer tensors out of its index file, so a loader that trusts the index loses long-context retrieval. The install script adds them and checks all 29 files against the lock. The trade-offs are single-request serving, no speculative decoding and some quality lost against the BF16 original, with no size given. ```flow Fetch the 3.05 bpw EXL3 weights Add 22 indexer entries to the index Check all 29 files against the lock Build the image and run preflight Serve an OpenAI-compatible API ``` Good to know: The timings below come from the card's benchmark on one machine, one request at a time. ```chart Title: Seconds to the first token 131,072 prefill profile: 113.5 s 260,096 prefill profile: 235.3 s 131,072 agent profile: 236.2 s 8,192 agent profile: 9.5 s ``` - Hardware: validated on one machine only, with two RTX 3090s, a Threadripper PRO 5975WX and 128 GB of DDR4-3200 memory. - Speed: decode depends heavily on CPU memory bandwidth, and prefill depends on PCIe bandwidth. - Real agent turns: decoded at 37.7 tokens per second on average. - Cache: across 15 agent tasks, prefix caching served 72.9% of the prompt tokens. - License: MIT for GLM-5.3-Flash and for turboderp's quantization, Apache-2.0 for the runtime. Review the upstream terms before use. - Cost to run: none named, no API key, account or paid service, apart from the hardware. - Data it reaches: it downloads the weights from Hugging Face and serves an API on your own machine. - Not for: several requests at once, speculative decoding, or machines without two 24 GB GPUs. ## On Hugging Face, 13 November 2026 - Task: text generation - Downloads in the last 30 days: 0 - Likes: 2 - License: mit - Based on: zai-org/GLM-5.3-Flash - Tags: glm-5.3, exl3, exllamav3, vllm, 256k-context, rtx-3090, dual-gpu, 24gb-vram, 128gb-ram, consumer-gpu, cpu-offload, local-llm ## Links - Model: https://huggingface.co/albucino/GLM-5.3-Flash-EXL3-3.05bpw - Files: https://huggingface.co/albucino/GLM-5.3-Flash-EXL3-3.05bpw/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download albucino/GLM-5.3-Flash-EXL3-3.05bpw ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [albucino/GLM-5.3-Flash-EXL3-3.05bpw on Hugging Face](https://huggingface.co/albucino/GLM-5.3-Flash-EXL3-3.05bpw) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | GLM-5.3-Flash EXL3 3.05bpw (this page) | 0 | 2 | — | mit | 2026-10-12 | | [DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000](https://mindket.com/p/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e) | 2,514 | 42 | 219B | mit | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017-EXL3-franken-v100](https://mindket.com/p/91bd8bd6-2fba-4896-ae7f-5c8ff314c941) | 313 | 7 | — | mit | 2026-10-26 | | [DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw](https://mindket.com/p/e2aa9049-7d40-46f6-b187-57f9ffaaccf1) | 8,113 | 46 | 105B | mit | 2026-10-20 | | [DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw](https://mindket.com/p/037efcc1-e5e0-465b-9448-56fd5a06fdc6) | 7,382 | 60 | 105B | mit | 2026-10-19 | | [Qwen3.8-27B iMatrix NVFP4 MTP GGUF](https://mindket.com/p/d44ebd5f-22b5-4277-81d0-197775776c4f) | 3,271,044 | 51 | 461M | apache-2.0 | 2026-08-17 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-14. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-12 - First published: 2026-10-12 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open albucino/GLM-5.3-Flash-EXL3-3.05bpw on Hugging Face](https://huggingface.co/albucino/GLM-5.3-Flash-EXL3-3.05bpw) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/783914ac-53d0-4c6b-adb4-69db026aa354) - [Its page on Mindket](https://mindket.com/p/783914ac-53d0-4c6b-adb4-69db026aa354)