Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

A coding-focused GGUF build of Qwen3.8-Flash-Next with half its experts removed, cutting 354 GB at BF16 to 58.4 GB while keeping image input. For people with large machines who want a local coder.

Made by ISTA-DASLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds a GGUF build of Qwen3.8-Flash-Next that its makers at ISTA-DASLab shrank for coding. They cut half of the experts from the mixture-of-experts model and stored the rest in a compact format, so the files total 58.4 GB instead of 354 GB at BF16. The cut aims at code, tool use by agents, vision and spatial reasoning, and image input stays. The card calls it an experimental release.

Who it suits: It suits people with a large machine who want a coding-focused model they can run locally, and who accept weaker results outside that focus.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a coding-focused model locally | A private code assistant on your own hardware
AI research | Researchers | Study expert pruning and low-bit storage | Compare the pruned model with the BF16 base
Infrastructure | ML engineers | Fit a huge model onto less hardware | Serve a compressed model on a 32 GB accelerator
Local AI | Hobbyists with big machines | Try a frontier-scale coder at home | Download the GGUF files and test them on coding tasks

If you need a general-purpose model, you can skip it, since the card points to the unpruned release for that.

Where to use it:

• Tools that load GGUF files, such as llama.cpp
• Coding and agent setups that send tasks to a local model
• Windows, macOS or Linux machines with plenty of memory

Use case | Who | What they get back
Code generation | Developers | Code from a model whose kept experts favor coding
Agent tool use | Agent builders | Tool calls and multi-step coding work
Image questions | Developers | Text answers about pictures, since the task is image and text to text

How to set it up: You need these first:

• Python, for the Hugging Face download command
• About 58.4 GB of disk for the files
• Enough memory, since the card says 29.6 GB must stay resident while the n-gram shard may be served from disk
• A tool that loads GGUF files

1. Install the Hugging Face hub tool.
2. Download the model files.
3. Open them in a GGUF-capable tool.
4. If you have an earlier build, re-download the first shard.

pip install -U huggingface_hub
hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

How to use it: The part of the card given here shows no run commands, so read the full card for the run steps. The card says every score was measured at xhigh reasoning effort, so use that setting if you want to match them.

Tool | What it does
GGUF files | The compressed model, split into shards
n-gram shard | A lookup table that may be served from disk
xhigh reasoning effort | The setting the card used for its scores
Expert pruning | Removes half of the experts to cut the size

Example: You load the model in a GGUF tool, paste a bug report and ask for a fix. It answers with code, and in an agent setup it can call tools. The card does not show this exact run, so treat it as an expectation.

For newcomers and experts: A mixture-of-experts layer holds many small blocks, and a router picks a few for each token, so only part of the model runs at a time. Pruning removes half of those blocks and quantizing stores the rest in fewer bits. A good first step is to check your memory against the file sizes before you download.

The base model uses 512 experts in each of its 48 layers, and 10 of them run for each token. The kept weights use 3.5 bits per weight. The figure of 1.89 bits per parameter is an effective rate over the original model, and no single weight is stored at that rate.

Start from Qwen3.8-Flash-Next
Remove half of the experts, aimed at code
Store the kept weights at 3.5 bits
Load the GGUF files and run them

Good to know: The size drop is large, and the card is open that some abilities are lost.

Title: Model size in GB
BF16 base: 354 GB
This release: 58.4 GB

• Scores: on SWE-bench Verified the BF16 base scores 82.80 and this release 79.20, which keeps 95.7%, and LiveCodeBench v6 stays at 87.43, all by the authors at xhigh effort
• Trade-off: abilities outside code, tool use, vision and spatial reasoning drop, and the card points to unpruned releases for general use
• Update: a later build reduces looping and raises coding scores, so re-download the first shard if you have the earlier one
• Other versions: 10 quantized and 1 fine-tuned version made by other people are listed
• License: Apache 2.0
• Cost to run: none named, you supply the hardware
• Data it reaches: the prompts and images you give it, and the files you download

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 117B
• Weights: about 58.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 781,625
• Likes: 610
• License: apache-2.0
• Based on: Qwen/Qwen3.8-Flash-Next
• Quantized versions: 10
• Fine-tunes: 1
• Tags: gguf, quantization, pruning, expert-pruning, mixed-precision, ist-daslab, code, image-text-to-text, imatrix, conversational

Links

• Model: https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF
• Files: https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.