--- title: "Qwen3.8-2B-Distill-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/e7e883a1-dae8-4084-aaee-c3e51741cb1b" updated: "2026-11-12" --- # Qwen3.8-2B-Distill-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a small 2B reasoning model on phones, laptops and single-board computers, as GGUF files in five quant levels. A distillation of a large Qwen3.8 model, packaged by Empero for llama.cpp. Qwen3.8-2B-Distill-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by empero-ai ## About ### What it is This repository holds GGUF files of Qwen3.8-2B from empero-ai, a small reasoning model with about 1.9B parameters. The model was distilled from a much larger Qwen3.8 teacher into the Qwen3.5-2B architecture. The files come in five sizes, from a 1.312 GB Q4\_K\_M file to a 3.897 GB BF16 reference. ### Who it suits People who want a small local model for chat, math and reasoning on modest hardware, and developers who need something light for apps. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Embed a small local model in an app | A chat feature that works offline | | Education | Students and teachers | Try a reasoning model on a school laptop | A math helper that runs locally | | Hobby electronics | Makers | Run a model on a single-board computer | A home assistant prototype | | Research | ML researchers | Study a distilled small model against its base | A comparison on math word problems | If you want the largest and most capable model, you can skip it, since this one is built for small devices. ### Where to use it - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Other stock GGUF runtimes on Windows, macOS or Linux - Phones and single-board computers, with the smaller quants | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Individuals | Reasoned answers produced on a phone or laptop | | Math and logic | Students | Worked solutions with a visible thinking block | | Offline assistants | Developers | A small model without a network connection | ### How to set it up - Python, to run the Hugging Face download tool - A llama.cpp release new enough to know the Qwen3.5 design and its Gated DeltaNet layers, because older ones cannot open the file - Q4\_K\_M and Q5\_K\_M suit phones and any modern laptop, even CPU-only - Q6\_K and Q8\_0 need a GPU with 4 GB or more, or a CPU with 8 GB of RAM - BF16 needs a GPU with 6 GB or more 1. Install the Hugging Face hub tools with pip. 2. Download the repository with the hf command, or pick one GGUF file. 3. Load the file in llama.cpp, Ollama, LM Studio, Jan or KoboldCpp. ```bash pip install -U huggingface_hub hf download empero-ai/Qwen3.8-2B-Distill-GGUF ``` ### How to use it Start a chat with llama-cli and the file you chose. Use the built-in chat template, which the -cnv flag turns on, and allow a long output since the model thinks before it answers. ```bash llama-cli -m Qwen3.8-2B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` | Tool | What it does | | --- | --- | | Qwen3.8-2B-Q4\_K\_M.gguf | The recommended balance of quality and size, 1.312 GB | | Qwen3.8-2B-Q5\_K\_M.gguf | Higher quality for a modest size increase, 1.455 GB | | Qwen3.8-2B-Q6\_K.gguf | Close to lossless, 1.606 GB | | Qwen3.8-2B-Q8\_0.gguf | The highest-quality quantization, 2.077 GB | | Qwen3.8-2B-BF16.gguf | The full precision reference, 3.897 GB | ### Example You start llama-cli with the Q4\_K\_M file and ask "What is 17 times 23, and show your steps". The model first writes out its thinking, then gives the answer. Your app should hide that thinking block before showing the reply to users. ### For newcomers and experts Begin with the Q4\_K\_M file, which the card recommends, and keep the card's sampling values. If answers get cut off, raise the output limit. The base family mixes three Gated DeltaNet layers with each full-attention layer. The card says the distillation used about 30,000 curated teacher traces. At long context the KV cache becomes the main memory cost, so the weight size understates what you need. Download one GGUF file → Load it in a GGUF runtime → Set low temperature, top-p, top-k → Model thinks, then answers → Strip the thinking block for users ### Good to know - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own device - Not for: older llama.cpp builds, which fail to load this architecture Chain-of-thought scores in the card, base versus distilled | Label | Value | | --- | --- | | mmlu, Qwen3.5-2B base | 0.283 | | mmlu, Qwen3.8-2B | 0.548 | | gsm8k\_cot, Qwen3.5-2B base | 0.330 | | gsm8k\_cot, Qwen3.8-2B | 0.640 | ## README ````markdown # Qwen3.8-2B-Distill-GGUF Runs a small 2B reasoning model on phones, laptops and single-board computers, as GGUF files in five quant levels. A distillation of a large Qwen3.8 model, packaged by Empero for llama.cpp. Made by empero-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF files of Qwen3.8-2B from empero-ai, a small reasoning model with about 1.9B parameters. The model was distilled from a much larger Qwen3.8 teacher into the Qwen3.5-2B architecture. The files come in five sizes, from a 1.312 GB Q4_K_M file to a 3.897 GB BF16 reference. Who it suits: People who want a small local model for chat, math and reasoning on modest hardware, and developers who need something light for apps. ```table Industry | Who uses it | How they use it | Example Software | Developers | Embed a small local model in an app | A chat feature that works offline Education | Students and teachers | Try a reasoning model on a school laptop | A math helper that runs locally Hobby electronics | Makers | Run a model on a single-board computer | A home assistant prototype Research | ML researchers | Study a distilled small model against its base | A comparison on math word problems ``` If you want the largest and most capable model, you can skip it, since this one is built for small devices. Where to use it: - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Other stock GGUF runtimes on Windows, macOS or Linux - Phones and single-board computers, with the smaller quants ```table Use case | Who | What they get back Local chat | Individuals | Reasoned answers produced on a phone or laptop Math and logic | Students | Worked solutions with a visible thinking block Offline assistants | Developers | A small model without a network connection ``` How to set it up: - Python, to run the Hugging Face download tool - A llama.cpp release new enough to know the Qwen3.5 design and its Gated DeltaNet layers, because older ones cannot open the file - Q4_K_M and Q5_K_M suit phones and any modern laptop, even CPU-only - Q6_K and Q8_0 need a GPU with 4 GB or more, or a CPU with 8 GB of RAM - BF16 needs a GPU with 6 GB or more 1. Install the Hugging Face hub tools with pip. 2. Download the repository with the hf command, or pick one GGUF file. 3. Load the file in llama.cpp, Ollama, LM Studio, Jan or KoboldCpp. ```bash pip install -U huggingface_hub hf download empero-ai/Qwen3.8-2B-Distill-GGUF ``` How to use it: Start a chat with llama-cli and the file you chose. Use the built-in chat template, which the -cnv flag turns on, and allow a long output since the model thinks before it answers. ```bash llama-cli -m Qwen3.8-2B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` ```table Tool | What it does Qwen3.8-2B-Q4_K_M.gguf | The recommended balance of quality and size, 1.312 GB Qwen3.8-2B-Q5_K_M.gguf | Higher quality for a modest size increase, 1.455 GB Qwen3.8-2B-Q6_K.gguf | Close to lossless, 1.606 GB Qwen3.8-2B-Q8_0.gguf | The highest-quality quantization, 2.077 GB Qwen3.8-2B-BF16.gguf | The full precision reference, 3.897 GB ``` Example: You start llama-cli with the Q4_K_M file and ask "What is 17 times 23, and show your steps". The model first writes out its thinking, then gives the answer. Your app should hide that thinking block before showing the reply to users. For newcomers and experts: Begin with the Q4_K_M file, which the card recommends, and keep the card's sampling values. If answers get cut off, raise the output limit. The base family mixes three Gated DeltaNet layers with each full-attention layer. The card says the distillation used about 30,000 curated teacher traces. At long context the KV cache becomes the main memory cost, so the weight size understates what you need. ```flow Download one GGUF file Load it in a GGUF runtime Set low temperature, top-p, top-k Model thinks, then answers Strip the thinking block for users ``` Good to know: - License: Apache 2.0 - Cost to run: none named - Data it reaches: downloads files from Hugging Face, then runs on your own device - Not for: older llama.cpp builds, which fail to load this architecture ```chart Title: Chain-of-thought scores in the card, base versus distilled mmlu, Qwen3.5-2B base: 0.283 mmlu, Qwen3.8-2B: 0.548 gsm8k_cot, Qwen3.5-2B base: 0.330 gsm8k_cot, Qwen3.8-2B: 0.640 ``` ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 1.9B - Weights: about 971 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 509,820 - Likes: 184 - License: apache-2.0 - Languages: en - Based on: empero-ai/Qwen3.8-2B - Quantized versions: 1 - Tags: gguf, llama.cpp, quantized, empero-ai, qwen3.5, qwen3.8, distillation, reasoning, gated-deltanet, edge, text-generation, conversational ## Links - Model: https://huggingface.co/empero-ai/Qwen3.8-2B-Distill-GGUF - Files: https://huggingface.co/empero-ai/Qwen3.8-2B-Distill-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download empero-ai/Qwen3.8-2B-Distill-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [empero-ai/Qwen3.8-2B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-2B-Distill-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-2B-Distill-GGUF (this page) | 509,820 | 184 | 1.9B | apache-2.0 | 2026-08-16 | | [Qwen3.8-4B-Distill-GGUF](https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30) | 570,974 | 208 | 4.3B | apache-2.0 | 2026-08-16 | | [Qwen3.8-9B-Distill-GGUF](https://mindket.com/p/cfc3041d-4b8a-4012-b704-ce834019b236) | 537,695 | 356 | 9.2B | apache-2.0 | 2026-08-16 | | [Qwen3.8-35B-A3B-Distill-GGUF](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 | | [Qwen3.8-27B-Ridge-GGUF](https://mindket.com/p/f254bd76-fee2-4e04-80ac-a73f9621c088) | 292,932 | 369 | 27.3B | apache-2.0 | 2026-08-15 | | [Qwen3.8-9B-Distill](https://mindket.com/p/f257e0b9-f595-4d2e-8a06-68cc42f90540) | 6,592 | 237 | 9.7B | apache-2.0 | 2026-08-15 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-08-16 - First published: 2026-08-15 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open empero-ai/Qwen3.8-2B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-2B-Distill-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e7e883a1-dae8-4084-aaee-c3e51741cb1b) - [Its page on Mindket](https://mindket.com/p/e7e883a1-dae8-4084-aaee-c3e51741cb1b)