MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

Gives you a 9B coding and tool-calling model as one 3.64 GiB GGUF file, quantized at about 3.5 bits per weight for running on your own machine.

Made by sainez on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: The author compressed a 9B Xiaomi MiMo model into one GGUF file that mixes bit widths, taking it from 16.68 GiB in BF16 to 3.64 GiB. Two methods named GSQ and RCO did the work. The aim is to keep agentic coding, short reasoning and accurate tool calls for long-context local use.

Who it suits: It suits people who run coding models on their own hardware, especially devices with unified memory, and researchers who study quantization.

Industry | Who uses it | How they use it | Example
Software development | Developers | Load the GGUF file in a local runtime for coding help | Fixing a bug with a model that stays on your machine
Local AI | Owners of unified-memory devices | Run a 9B model in about 3.64 GiB | Trying a coding agent without a hosted service
AI research | Quantization researchers | Study how mixed precision was assigned per tensor | Comparing this recipe with plain quantization

You can skip it if you need the full BF16 quality or a hosted service, since this is a compressed copy.

Where to use it:

• Local runtimes that read GGUF files
• Windows, macOS or Linux, with Python for the download step
• Devices with unified memory, which the card names as its target

Use case | Who | What they get back
Local coding help | Developers | Code answers with short reasoning from a model on your machine
Tool calling | Agent builders | Multi-turn tool calls, which the card says it aims to keep accurate
Quantization study | Researchers | A worked example of GSQ plus RCO on a hybrid model

How to set it up:

• Python, to run the Hugging Face download tool
• The huggingface_hub package
• A runtime that can load GGUF files
• Room for about 3.64 GiB of model file

1. Install the Hugging Face hub package.
2. Download the repository files to your computer.
3. Open the gguf file in your GGUF runtime.

pip install -U huggingface_hub
hf download sainez/MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

How to use it: The download holds the gguf model file and a small json receipt of each tensor's format. The card text shown here gives no run command, so load the gguf file the way your runtime documents.

Tool | What it does
hf download | Fetches the repository files to your computer
The gguf file | The quantized model itself
The json file | A receipt of each tensor's format and byte size

Example: You load the gguf file and paste a short function with a bug, asking for a fix. The card says the model aims to keep its thinking short inside think tags and to call tools accurately across turns.

For newcomers and experts: A quantized model is a shrunk copy that stores each number with fewer bits, so it fits on smaller machines. To start, run the two download commands and check that the gguf file is about 3.64 GiB.

If you know the field, the base model is a 32-layer hybrid with 24 Gated DeltaNet blocks and 8 Gated Attention blocks. GSQ refines grid codes and scales for the low-bit formats. RCO then assigns a format to each tensor under a fixed byte ceiling.

The BF16 checkpoint is the starting point
Each block matrix gets candidate formats
GSQ refines the low-bit candidates
RCO picks one format per tensor
The result is one GGUF file

Good to know:

Title: File size in GiB
BF16 original: 16.68 GiB
This quantization: 3.64 GiB

• Method: the two token embedding matrices stay at Q4_K, which the card says cuts vocabulary KL divergence by 17.0%
• Calibration: 2,048-token windows over coding, code reasoning, tools, reasoning and general text
• License: apache-2.0
• Cost to run: none named, it runs on your own hardware
• Data it reaches: the prompts you give it, and the files you download from Hugging Face
• Not for: matching the BF16 original exactly, since it is a quantized copy

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 9B
• Weights: about 4.5 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 1,548
• Likes: 2
• License: apache-2.0
• Based on: XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
• Tags: gguf, llama.cpp, quantization, qwen3_5, mimo, agentic-coding, text-generation, imatrix, conversational

Links

• Model: https://huggingface.co/sainez/MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF
• Files: https://huggingface.co/sainez/MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download sainez/MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.