Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF

Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF

Squeezes the Qwen3.8-Flash-Next mixture-of-experts model into about 37 GiB of text-only GGUF files for experiments. It needs a patched llama.cpp build that comes with the release.

Made by 0xKitkat on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is an experimental, text-only GGUF build of Qwen3.8-Flash-Next, a model with 125B total parameters and about 6B active. The author shrank Unsloth's 72.55 GB source file to 39,721,239,200 bytes, about 37 GiB. It uses mixed quantization by tensor role and needs a patched llama.cpp to load.

Who it suits: It suits tinkerers and engineers who enjoy pushing quantization to extremes and are comfortable building a patched runtime.

Industry | Who uses it | How they use it | Example
Local AI | Enthusiasts who run large models at home | Run a 125B mixture-of-experts model from a 37 GiB file | Load the three shards in the patched runtime and try a prompt
Quantization research | Engineers studying low-bit formats | Study mixed 1-bit and 2-bit tensors | Read how the first and last layers keep 2-bit expert banks
Inference tooling | llama.cpp contributors | Test the padded per-layer embedding layout | Check how 160-value rows are padded to 256 values

You can skip it if you need stock llama.cpp, Ollama or vision input, since this build is text-only and patched.

Where to use it:

• The pinned Qwen4Exp llama.cpp revision with the included patch applied
• Windows, macOS and Linux with Python, to download the files
• A machine that can hold a file of about 37 GiB

Use case | Who | What they get back
Local text generation | Enthusiasts | Replies from a heavily compressed 125B model
Quantization study | Researchers | A worked example of per-role mixed quantization
Runtime testing | Tool builders | A real model that uses the padded embedding layout

How to set it up: You need these first:

• Python, to use Hugging Face's download client
• Disk space for about 39.7 GB of GGUF shards
• The ability to build the pinned llama.cpp revision with the patch

1. Install or update the Hugging Face download client.
2. Download this repository, which holds the three GGUF shards, the patch and the build manifest.
3. Read the build manifest for the pinned llama.cpp revision, apply the patch to it, and build the runtime.
4. Load the shards with that patched runtime and send a text prompt.

pip install -U huggingface_hub
hf download 0xKitkat/Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF

How to use it: Once the patched runtime is built, you load the three shards and prompt the model with text only. The visible card text does not give a run command, so use the model page for the exact call.

Tool | What it does
patches/qwen4exp-under40.patch | Teaches the pinned llama.cpp revision to read the padded row layout
build-manifest.txt | Pins the exact build and runtime revision
SHA256SUMS | Holds the checksums for the shards
hf download | Fetches the repository's files to your computer

Example: You load the three shards in the patched runtime and send a short prompt. The card's own smoke test evaluated a prompt, produced a first token and exited with code 0, so expect text back, not a quality guarantee.

For newcomers and experts: This is a hard model to run, because stock tools cannot open it. A first small step is to read the model page and the build manifest before you spend time on the 37 GiB download.

For readers who know the field, the per-layer token table is padded from 160 to 256 values per row and stored as Q1_0. Expert down banks stay at Q2_0, and gate and up banks use Q1_0 inside and Q2_0 in layers 0 to 2 and 44 to 47. Attention, routers and embeddings are copied unchanged. The card reports 1.80 bits per weight and admits quantization error.

Download the shards and the patch
Apply the patch to the pinned llama.cpp
Build the patched runtime
Load the three shards
Send a text prompt

Good to know:

• Experimental: the card calls it an experimental build, and its visible text gives a smoke test but no quality benchmarks.
• Runtime: stock llama.cpp does not understand the custom row layout, so the included patch is required.
• Scope: text only. The vision projector and the separate speculation head are not included.
• Counts: Hugging Face lists 208B parameters and 104 GB at 4-bit, while the card says 125B and 37 GiB.

Title: File size
Source GGUF: 72.55 GB
UltraLite build: 39.721239 GB

• License: Apache 2.0.
• Cost to run: none named. You supply the hardware and build the patched runtime.
• Data it reaches: only the Hugging Face download of the files, as far as the source shows.
• Not for: stock llama.cpp or image input.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 208B
• Weights: about 104 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 340
• Likes: 11
• License: apache-2.0
• Based on: unsloth/Qwen3.8-Flash-Next-GGUF
• Tags: gguf, qwen, qwen4exp, llama.cpp, quantized, ultra-low-bit, text-generation, imatrix, conversational

Links

• Model: https://huggingface.co/0xKitkat/Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF
• Files: https://huggingface.co/0xKitkat/Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download 0xKitkat/Qwen3.8-Flash-Next-125B-UltraLite-37GiB-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.