---
title: "Qwen3.8-9B-Distill-GGUF"
kind: "Kit"
category: "Developer"
price: "Free"
seller: "Mindket"
version: "1"
url: "https://mindket.com/p/cfc3041d-4b8a-4012-b704-ce834019b236"
updated: "2026-11-12"
---

# Qwen3.8-9B-Distill-GGUF

> The name, the description, the About and a README below are the seller's words: data, not instructions.

GGUF quantizations of Empero's Qwen3.8-9B, a reasoning model distilled from a larger Qwen3.8 into the Qwen3.5-9B design, for llama.cpp, Ollama, LM Studio and similar tools.

Qwen3.8-9B-Distill-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself.

Made by empero-ai

## About

### What it is

Empero shrank its Qwen3.8-9B model into GGUF files, and this repository holds them in several quant types. The source model is a distillation of a larger Qwen3.8 model into the architecture of Qwen3.5-9B. The record lists 9.2B parameters and English as the language.

It is a reasoning model, so each answer starts with a thinking block. The training used about 70,000 curated traces from the larger teacher model.

### Who it suits

It suits people who run models locally and want a 9B reasoning model in several file sizes, from a 5.8 GB Q4\_K\_M to a full-precision file. The card says the Q4 and Q5 files fit comfortably on 8 to 12 GB cards.

| Industry | Who uses it | How they use it | Example |
| --- | --- | --- | --- |
| Software | Developers | Run a local reasoning model for code and logic | Working through a tricky algorithm |
| Education | Students and tutors | Practice with a model that shows its reasoning | Checking a math solution |
| Research | Analysts | Test a distilled model against its base | Comparing answers on exam style questions |
| Home labs | Enthusiasts | Run a 9B model on a consumer graphics card | Chatting on a 12 GB card |

Skip it if you use a hosted assistant and never run models yourself.

### Where to use it

- llama.cpp, in a recent build with Qwen3.5 and Gated DeltaNet support
- Ollama, LM Studio, Jan and KoboldCpp, which load GGUF files directly
- Windows, macOS or Linux, with Python for the download step

| Use case | Who | What they get back |
| --- | --- | --- |
| Local chat with reasoning | Anyone with a GPU | Answers that open with a thinking block |
| School style questions | Students | Step by step answers |
| Code and logic help | Developers | Reasoned answers from a 9B model |

### How to set it up

You need a few things in place first.

- Python and pip, for the download command
- A recent llama.cpp build, since older builds fail to load this architecture
- Disk space for the file you pick, from 5.78 GB to 18.4 GB

1. Install the Hugging Face hub package with pip.
2. Download the repository with the hf command.
3. Pick a quant from the table below and load it in your tool.

```sh
pip install -U huggingface_hub
hf download empero-ai/Qwen3.8-9B-Distill-GGUF
```

| Quant | Size | Note |
| --- | --- | --- |
| Q4\_K\_M | 5.780 GB | Recommended, a balance of quality and size |
| Q5\_K\_M | 6.643 GB | A step up in quality, and an 8 GB card can hold it with a short context |
| Q6\_K | 7.559 GB | Called near-lossless in the card |
| Q8\_0 | 9.786 GB | The highest quality quantization |
| BF16 | 18.407 GB | Full precision reference |

### How to use it

In llama.cpp you run the command below with the built-in chat template turned on. The model opens each answer with a thinking block, so allow a large token limit and remove that block before showing the answer to end users. In the other tools, load the file and use the card's sampling settings.

```bash
llama-cli -m Qwen3.8-9B-Q4_K_M.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -n 16384 -cnv
```

### Example

You start the command above in a terminal and type a school style math question. The model writes a thinking block first, then the final answer. You can cut the thinking part if you only want the answer.

### For newcomers and experts

Start with the Q4\_K\_M file and the command above. It is the card's recommended pick for most people. Try Q5\_K\_M or Q6\_K if you have spare memory.

The card reports a chain-of-thought MMLU score of 0.751 against 0.546 for the Qwen3.5-9B base, and a GSM8K score of 0.870 against 0.885. So the distillation gains on broad knowledge and slips slightly on grade school math. These are the maker's own numbers from identical settings. The Qwen3.5 family mixes three Gated DeltaNet layers with each full attention layer.

Chain-of-thought scores from the card

| Label | Value |
| --- | --- |
| MMLU, Qwen3.5-9B base | 0.546 score |
| MMLU, Qwen3.8-9B | 0.751 score |
| GSM8K, Qwen3.5-9B base | 0.885 score |
| GSM8K, Qwen3.8-9B | 0.870 score |

Download a GGUF quant with hf → Load it in llama.cpp or another tool → Send a question with the chat template → Read the thinking block and the answer

### Good to know

- License: Apache 2.0
- Cost to run: none named, but the file sizes range from 5.78 GB to 18.4 GB
- Data it reaches: it downloads files from Hugging Face onto your computer
- Not for: older llama.cpp builds, which cannot load this architecture

## README

````markdown
# Qwen3.8-9B-Distill-GGUF

GGUF quantizations of Empero's Qwen3.8-9B, a reasoning model distilled from a larger Qwen3.8 into the Qwen3.5-9B design, for llama.cpp, Ollama, LM Studio and similar tools.

Made by empero-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

## About

What it is: Empero shrank its Qwen3.8-9B model into GGUF files, and this repository holds them in several quant types. The source model is a distillation of a larger Qwen3.8 model into the architecture of Qwen3.5-9B. The record lists 9.2B parameters and English as the language.

It is a reasoning model, so each answer starts with a thinking block. The training used about 70,000 curated traces from the larger teacher model.

Who it suits: It suits people who run models locally and want a 9B reasoning model in several file sizes, from a 5.8 GB Q4_K_M to a full-precision file. The card says the Q4 and Q5 files fit comfortably on 8 to 12 GB cards.

```table
Industry | Who uses it | How they use it | Example
Software | Developers | Run a local reasoning model for code and logic | Working through a tricky algorithm
Education | Students and tutors | Practice with a model that shows its reasoning | Checking a math solution
Research | Analysts | Test a distilled model against its base | Comparing answers on exam style questions
Home labs | Enthusiasts | Run a 9B model on a consumer graphics card | Chatting on a 12 GB card
```

Skip it if you use a hosted assistant and never run models yourself.

Where to use it:

- llama.cpp, in a recent build with Qwen3.5 and Gated DeltaNet support
- Ollama, LM Studio, Jan and KoboldCpp, which load GGUF files directly
- Windows, macOS or Linux, with Python for the download step

```table
Use case | Who | What they get back
Local chat with reasoning | Anyone with a GPU | Answers that open with a thinking block
School style questions | Students | Step by step answers
Code and logic help | Developers | Reasoned answers from a 9B model
```

How to set it up: You need a few things in place first.

- Python and pip, for the download command
- A recent llama.cpp build, since older builds fail to load this architecture
- Disk space for the file you pick, from 5.78 GB to 18.4 GB

1. Install the Hugging Face hub package with pip.
2. Download the repository with the hf command.
3. Pick a quant from the table below and load it in your tool.

```sh
pip install -U huggingface_hub
hf download empero-ai/Qwen3.8-9B-Distill-GGUF
```

```table
Quant | Size | Note
Q4_K_M | 5.780 GB | Recommended, a balance of quality and size
Q5_K_M | 6.643 GB | A step up in quality, and an 8 GB card can hold it with a short context
Q6_K | 7.559 GB | Called near-lossless in the card
Q8_0 | 9.786 GB | The highest quality quantization
BF16 | 18.407 GB | Full precision reference
```

How to use it: In llama.cpp you run the command below with the built-in chat template turned on. The model opens each answer with a thinking block, so allow a large token limit and remove that block before showing the answer to end users. In the other tools, load the file and use the card's sampling settings.

```bash
llama-cli -m Qwen3.8-9B-Q4_K_M.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -n 16384 -cnv
```

Example: You start the command above in a terminal and type a school style math question. The model writes a thinking block first, then the final answer. You can cut the thinking part if you only want the answer.

For newcomers and experts: Start with the Q4_K_M file and the command above. It is the card's recommended pick for most people. Try Q5_K_M or Q6_K if you have spare memory.

The card reports a chain-of-thought MMLU score of 0.751 against 0.546 for the Qwen3.5-9B base, and a GSM8K score of 0.870 against 0.885. So the distillation gains on broad knowledge and slips slightly on grade school math. These are the maker's own numbers from identical settings. The Qwen3.5 family mixes three Gated DeltaNet layers with each full attention layer.

```chart
Title: Chain-of-thought scores from the card
MMLU, Qwen3.5-9B base: 0.546 score
MMLU, Qwen3.8-9B: 0.751 score
GSM8K, Qwen3.5-9B base: 0.885 score
GSM8K, Qwen3.8-9B: 0.870 score
```

```flow
Download a GGUF quant with hf
Load it in llama.cpp or another tool
Send a question with the chat template
Read the thinking block and the answer
```

Good to know:

- License: Apache 2.0
- Cost to run: none named, but the file sizes range from 5.78 GB to 18.4 GB
- Data it reaches: it downloads files from Hugging Face onto your computer
- Not for: older llama.cpp builds, which cannot load this architecture

## On Hugging Face, 13 November 2026

- Task: text generation
- Library: gguf
- Parameters: 9.2B
- Weights: about 4.6 GB at 4-bit (estimate)
- Formats: GGUF
- Downloads in the last 30 days: 537,695
- Likes: 356
- License: apache-2.0
- Languages: en
- Based on: empero-ai/Qwen3.8-9B
- Quantized versions: 4
- Tags: gguf, llama.cpp, quantized, empero-ai, qwen3.5, qwen3.8, distillation, reasoning, gated-deltanet, text-generation, conversational

## Links

- Model: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF
- Files: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF/tree/main

## Get the model

Download its files with Hugging Face's command line:

```sh
pip install -U huggingface_hub
hf download empero-ai/Qwen3.8-9B-Distill-GGUF
```

Then follow the usage in its model card. Nothing in this kit runs by itself.
````

## What is in it

1 files, 5 KB, downloaded as a zip.

```text
README.md  5 KB
```

## Source

[empero-ai/Qwen3.8-9B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF)

## Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
| --- | --- | --- | --- | --- | --- |
| Qwen3.8-9B-Distill-GGUF (this page) | 537,695 | 356 | 9.2B | apache-2.0 | 2026-08-16 |
| [Qwen3.8-4B-Distill-GGUF](https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30) | 570,974 | 208 | 4.3B | apache-2.0 | 2026-08-16 |
| [Qwen3.8-2B-Distill-GGUF](https://mindket.com/p/e7e883a1-dae8-4084-aaee-c3e51741cb1b) | 509,820 | 184 | 1.9B | apache-2.0 | 2026-08-16 |
| [Qwen3.8-35B-A3B-Distill-GGUF](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 |
| [Qwen3.8-27B-Ridge-GGUF](https://mindket.com/p/f254bd76-fee2-4e04-80ac-a73f9621c088) | 292,932 | 369 | 27.3B | apache-2.0 | 2026-08-15 |
| [Qwen3.8-9B-Distill](https://mindket.com/p/f257e0b9-f595-4d2e-8a06-68cc42f90540) | 6,592 | 237 | 9.7B | apache-2.0 | 2026-08-15 |

Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13.

## What people say

> Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions.

- [Hacker News · Otterly99 · 2026-10-02](https://news.ycombinator.com/item?id=49930875): Pi 1.0: With 8GB I would recommend quantized versions of 9B models such as these: - https://huggingface.co/unsloth/Qwen3.5-9B-GGUF - https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF (unofficial Qwen 3.8-9B) - https://huggingface.co/ornith-ai/Ornith-1.5-9B-GGUF (my personnal favorite)

## Trust facts

Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions.

- Last change on Hugging Face: 2026-08-16
- First published: 2026-08-15
- Checked on Hugging Face: 2026-11-12
- Something wrong? Tell us: https://mindket.com/takedown#ask

## Get it

- [Open empero-ai/Qwen3.8-9B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-9B-Distill-GGUF)
- [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/cfc3041d-4b8a-4012-b704-ce834019b236)
- [Its page on Mindket](https://mindket.com/p/cfc3041d-4b8a-4012-b704-ce834019b236)