DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

GGUF versions of DeepSeek-V4.1-Flash-1017 at three small sizes, with each tensor quantized at its own precision. For people running a huge mixture-of-experts model on limited hardware.

Made by ISTA-DASLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds three quantized GGUF builds of DeepSeek-V4.1-Flash-1017. The original is an expert-mixture model that totals 763.2B parameters, with only some experts active per token. Each tensor gets its own quantization type, chosen by a search that gives more bits to the sensitive parts within a total size budget. The files are standard GGUF.

Who it suits: It suits people who want to run a huge open model locally and accept low-bit quality in return for a small footprint.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists | Run a huge model on a single consumer GPU | Load the smallest build with Strata on a 12 GB card
Software | Developers | Serve a large model through llama.cpp | Use it as a local text generator for tooling
AI research | Quantization researchers | Study per-tensor bit allocation at low bit rates | Compare the three sizes against the base model
Infrastructure | ML engineers | Test how small a large model can get | Pick the size that fits a fixed memory budget

If you need near-lossless quality at higher precision, these files are not aimed at you.

Where to use it:

• llama.cpp build b11870 or newer, which reads the files and handles image input too
• Strata, which the card says runs every build on one GPU with 12 GB of VRAM
• Windows, macOS or Linux

Use case | Who | What they get back
Local text generation | Developers | Text from a 763.2B-parameter model on modest hardware
Low-memory experiments | Researchers | Three sizes to test against each other
Updating an old download | Earlier users | The current files, made locally without a full re-download

How to set it up: You need these first:

• Python, for the Hugging Face download command
• llama.cpp b11870 or newer
• Strata, if you want to use one 12 GB GPU

1. Install the Hugging Face hub tool and download the files.
2. For a new download, link the shared Engram file once for each build you use.
3. Load a build in llama.cpp or Strata.

pip install -U huggingface_hub
hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

Every build uses the same Engram file as its sixth shard. In the command below, replace the placeholder with the size name of your build.

ln -s DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf DeepSeek-V4.1-Flash-1017-GSQ-RCO-<ID>-00006-of-00006.gguf

How to use it: Load the build you picked in llama.cpp or in Strata and chat with it. People who already hold the older files can skip a second download.

Tool | What it does
GSQ | Learns the quantization grid and scales for each tensor
RCO | Chooses a quantization type for each tensor within a size budget
convert_ds41_upstream.py | Rewrites an earlier download locally into the current files and fetches only 80 KB
Strata | Makes each build usable on one GPU that has 12 GB of VRAM

Example: You pick the IQ2_XXS build, link the shared Engram file, and load it in llama.cpp. You paste a stack trace and ask what went wrong. The model answers in text, as any chat model would.

For newcomers and experts: A quantized model is a smaller copy of a big one that gives up some accuracy to save memory. A good first step is to try the smallest size and see whether its answers are good enough for you.

The model has 384 routed experts per layer, and 6 of them fire for each token. Its context window holds 1M tokens. The routed experts take most of the size budget, so the search sets a type per layer. GGUF cannot express a type per expert. GSQ makes each tensor accurate at a given type, and RCO allocates the types.

GSQ quantizes each tensor at a given type
RCO assigns a type to each tensor
The result is saved as standard GGUF
llama.cpp or Strata loads the build

Good to know: The three builds differ in bits per weight, and the card shows a plot comparing them with the base model on AIME25, GPQA-Diamond and LiveCodeBench v6.

Title: Bits per weight of each build
IQ1_M: 1.75 bits
IQ2_XXS: 2.00 bits
IQ2_XS: 2.25 bits

• Updating: people with older files can convert them in place instead of downloading again
• Other versions: 18 quantized and 3 fine-tuned versions made by other people are listed
• License: MIT
• Cost to run: none named, but even the small builds need a lot of memory
• Data it reaches: the model files you load

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 197B
• Weights: about 98.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 3,635,772
• Likes: 655
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Quantized versions: 20
• Fine-tunes: 3
• Tags: gguf, quantization, mixed-precision, ist-daslab, deepseek, text-generation, conversational

Links

• Model: https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF
• Files: https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.