--- title: "Qwen3.8-4B-Distill-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30" updated: "2026-11-12" --- # Qwen3.8-4B-Distill-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 4B reasoning model on a laptop or a mid-range graphics card, as GGUF files in five quant levels. A distillation of a large Qwen3.8 model, packaged by Empero for llama.cpp and similar apps. Qwen3.8-4B-Distill-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by empero-ai ## About ### What it is This repository holds GGUF files of Qwen3.8-4B from empero-ai, a reasoning model with about 4.3B parameters. It was distilled from a much larger Qwen3.8 teacher into the Qwen3.5-4B design. Five files are offered, from a 2.783 GB Q4\_K\_M quant to an 8.666 GB BF16 reference. ### Who it suits People who want a mid-size local model for chat, math and reasoning, and developers who need a model that fits on a consumer GPU. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a local model behind an app or a chat window | An offline assistant for a desktop tool | | Education | Students and teachers | Practice reasoning problems on a personal computer | A study helper that works offline | | Research | ML researchers | Compare a distilled model against its base | A test on knowledge and math sets | | IT operations | Small teams | Keep prompts on their own hardware | A private note summarizer | If you need a model for long documents or the best possible accuracy, you can skip it, since it is a small distilled model. ### Where to use it - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Other stock GGUF runtimes on Windows, macOS or Linux - Laptops and desktops with or without a graphics card | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Individuals | Reasoned answers produced on your own computer | | Knowledge questions | Students | Step-by-step answers with a visible thinking block | | App features | Developers | A model that runs without a network connection | ### How to set it up - Python, to run the Hugging Face download tool - A llama.cpp release new enough to know the Qwen3.5 design and its Gated DeltaNet layers, because older ones cannot open the file - For the two smaller quants, a card with 4 to 6 GB works, and CPU-only runs are also a strong option - For Q6\_K and Q8\_0, 6 to 8 GB is recommended, and BF16 wants 12 GB or more 1. Install the Hugging Face hub tools with pip. 2. Download the repository with the hf command, or pick one GGUF file. 3. Load the file in llama.cpp, Ollama, LM Studio, Jan or KoboldCpp. ```bash pip install -U huggingface_hub hf download empero-ai/Qwen3.8-4B-Distill-GGUF ``` ### How to use it Start a chat with llama-cli and the file you picked. The -cnv flag turns on the built-in chat template, and you should allow a long output because the model thinks before it answers. ```bash llama-cli -m Qwen3.8-4B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` | Tool | What it does | | --- | --- | | Qwen3.8-4B-Q4\_K\_M.gguf | The recommended balance of quality and size, 2.783 GB | | Qwen3.8-4B-Q5\_K\_M.gguf | Higher quality for a modest size increase, 3.161 GB | | Qwen3.8-4B-Q6\_K.gguf | Close to lossless, 3.563 GB | | Qwen3.8-4B-Q8\_0.gguf | The highest-quality quantization, 4.611 GB | | Qwen3.8-4B-BF16.gguf | The full precision reference, 8.666 GB | ### Example You start llama-cli with the Q4\_K\_M file and ask "Explain why the sky is blue in three steps". The model writes out its thinking first, then gives the three steps. Your app should hide the thinking block before showing the reply. ### For newcomers and experts Begin with the Q4\_K\_M file the card recommends and keep its sampling values. If answers stop early, raise the output limit. The card says the distillation used about 45,000 curated teacher traces. In its own table, the distilled model scores 0.553 on mmlu against 0.354 for the base, but 0.785 on gsm8k\_cot against 0.850. Long contexts make the KV cache the main memory cost, so weight size understates what you need. Download one GGUF file → Load it in a GGUF runtime → Set low temperature, top-p, top-k → Model thinks, then answers → Strip the thinking block for users ### Good to know - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own device - Not for: older llama.cpp builds, which cannot load this architecture - Limits: the card's table shows a lower gsm8k\_cot score than the base model Chain-of-thought scores in the card, base versus distilled | Label | Value | | --- | --- | | mmlu, Qwen3.5-4B base | 0.354 | | mmlu, Qwen3.8-4B | 0.553 | | gsm8k\_cot, Qwen3.5-4B base | 0.850 | | gsm8k\_cot, Qwen3.8-4B | 0.785 | ## README ````markdown # Qwen3.8-4B-Distill-GGUF Runs a 4B reasoning model on a laptop or a mid-range graphics card, as GGUF files in five quant levels. A distillation of a large Qwen3.8 model, packaged by Empero for llama.cpp and similar apps. Made by empero-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF files of Qwen3.8-4B from empero-ai, a reasoning model with about 4.3B parameters. It was distilled from a much larger Qwen3.8 teacher into the Qwen3.5-4B design. Five files are offered, from a 2.783 GB Q4_K_M quant to an 8.666 GB BF16 reference. Who it suits: People who want a mid-size local model for chat, math and reasoning, and developers who need a model that fits on a consumer GPU. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a local model behind an app or a chat window | An offline assistant for a desktop tool Education | Students and teachers | Practice reasoning problems on a personal computer | A study helper that works offline Research | ML researchers | Compare a distilled model against its base | A test on knowledge and math sets IT operations | Small teams | Keep prompts on their own hardware | A private note summarizer ``` If you need a model for long documents or the best possible accuracy, you can skip it, since it is a small distilled model. Where to use it: - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Other stock GGUF runtimes on Windows, macOS or Linux - Laptops and desktops with or without a graphics card ```table Use case | Who | What they get back Local chat | Individuals | Reasoned answers produced on your own computer Knowledge questions | Students | Step-by-step answers with a visible thinking block App features | Developers | A model that runs without a network connection ``` How to set it up: - Python, to run the Hugging Face download tool - A llama.cpp release new enough to know the Qwen3.5 design and its Gated DeltaNet layers, because older ones cannot open the file - For the two smaller quants, a card with 4 to 6 GB works, and CPU-only runs are also a strong option - For Q6_K and Q8_0, 6 to 8 GB is recommended, and BF16 wants 12 GB or more 1. Install the Hugging Face hub tools with pip. 2. Download the repository with the hf command, or pick one GGUF file. 3. Load the file in llama.cpp, Ollama, LM Studio, Jan or KoboldCpp. ```bash pip install -U huggingface_hub hf download empero-ai/Qwen3.8-4B-Distill-GGUF ``` How to use it: Start a chat with llama-cli and the file you picked. The -cnv flag turns on the built-in chat template, and you should allow a long output because the model thinks before it answers. ```bash llama-cli -m Qwen3.8-4B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` ```table Tool | What it does Qwen3.8-4B-Q4_K_M.gguf | The recommended balance of quality and size, 2.783 GB Qwen3.8-4B-Q5_K_M.gguf | Higher quality for a modest size increase, 3.161 GB Qwen3.8-4B-Q6_K.gguf | Close to lossless, 3.563 GB Qwen3.8-4B-Q8_0.gguf | The highest-quality quantization, 4.611 GB Qwen3.8-4B-BF16.gguf | The full precision reference, 8.666 GB ``` Example: You start llama-cli with the Q4_K_M file and ask "Explain why the sky is blue in three steps". The model writes out its thinking first, then gives the three steps. Your app should hide the thinking block before showing the reply. For newcomers and experts: Begin with the Q4_K_M file the card recommends and keep its sampling values. If answers stop early, raise the output limit. The card says the distillation used about 45,000 curated teacher traces. In its own table, the distilled model scores 0.553 on mmlu against 0.354 for the base, but 0.785 on gsm8k_cot against 0.850. Long contexts make the KV cache the main memory cost, so weight size understates what you need. ```flow Download one GGUF file Load it in a GGUF runtime Set low temperature, top-p, top-k Model thinks, then answers Strip the thinking block for users ``` Good to know: - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own device - Not for: older llama.cpp builds, which cannot load this architecture - Limits: the card's table shows a lower gsm8k_cot score than the base model ```chart Title: Chain-of-thought scores in the card, base versus distilled mmlu, Qwen3.5-4B base: 0.354 mmlu, Qwen3.8-4B: 0.553 gsm8k_cot, Qwen3.5-4B base: 0.850 gsm8k_cot, Qwen3.8-4B: 0.785 ``` ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 4.3B - Weights: about 2.2 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 570,974 - Likes: 208 - License: apache-2.0 - Languages: en - Based on: empero-ai/Qwen3.8-4B - Quantized versions: 2 - Tags: gguf, llama.cpp, quantized, empero-ai, qwen3.5, qwen3.8, distillation, reasoning, gated-deltanet, text-generation, conversational ## Links - Model: https://huggingface.co/empero-ai/Qwen3.8-4B-Distill-GGUF - Files: https://huggingface.co/empero-ai/Qwen3.8-4B-Distill-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download empero-ai/Qwen3.8-4B-Distill-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [empero-ai/Qwen3.8-4B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-4B-Distill-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-4B-Distill-GGUF (this page) | 570,974 | 208 | 4.3B | apache-2.0 | 2026-08-16 | | [Qwen3.8-9B-Distill-GGUF](https://mindket.com/p/cfc3041d-4b8a-4012-b704-ce834019b236) | 537,695 | 356 | 9.2B | apache-2.0 | 2026-08-16 | | [Qwen3.8-2B-Distill-GGUF](https://mindket.com/p/e7e883a1-dae8-4084-aaee-c3e51741cb1b) | 509,820 | 184 | 1.9B | apache-2.0 | 2026-08-16 | | [Qwen3.8-35B-A3B-Distill-GGUF](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 | | [Qwen3.8-27B-Ridge-GGUF](https://mindket.com/p/f254bd76-fee2-4e04-80ac-a73f9621c088) | 292,932 | 369 | 27.3B | apache-2.0 | 2026-08-15 | | [Qwen3.8-9B-Distill](https://mindket.com/p/f257e0b9-f595-4d2e-8a06-68cc42f90540) | 6,592 | 237 | 9.7B | apache-2.0 | 2026-08-15 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-08-16 - First published: 2026-08-15 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open empero-ai/Qwen3.8-4B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-4B-Distill-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/ef3614cf-441f-4388-b81f-7b72b3e7ed30) - [Its page on Mindket](https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30)