--- title: "Qwen3.8-35B-A3B-Distill-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7" updated: "2026-11-12" --- # Qwen3.8-35B-A3B-Distill-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 35B Mixture-of-Experts model with about 3B active parameters per token from GGUF files in ten sizes, plus a vision projector. For llama.cpp, Ollama, LM Studio, Jan and KoboldCpp users. Qwen3.8-35B-A3B-Distill-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by empero-ai ## About ### What it is This repository packs empero-ai's Qwen3.8-35B-A3B-Distill into GGUF files of different sizes. That model was distilled from the Qwen3.8 frontier models into a Qwen3.6-35B-A3B mixture-of-experts network. It has 35B parameters in total, and about 3B work on each token. You can load the files in llama.cpp, in Ollama, or in desktop apps such as LM Studio, Jan and KoboldCpp. ### Who it suits People who run large models on their own machine and want a file sized for their memory, including those with only a CPU and plenty of RAM. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software development | Local-model users | Run a reasoning model offline | A chat session in llama.cpp | | AI tooling | Desktop app users | Load the file in LM Studio, Jan or KoboldCpp | A local chat app using the Q4\_K\_M file | | Research | Model researchers | Compare quantization levels | Q2\_K against Q8\_0 on one model | | Vision work | Hobbyists | Describe images with the vision projector | A photo described through llama-mtmd-cli | If you have less than 16 GB of memory, you can skip it, since even the smallest file is about 12.6 GB. ### Where to use it - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Windows, macOS or Linux, after you fetch the files with Python - A machine with enough VRAM or system RAM to hold the whole file | Use case | Who | What they get back | | --- | --- | --- | | Reasoning chat | Local-model users | Answers that open with a think block | | Image description | Hobbyists | A description of a photo, using the vision projector | | CPU or partial offload runs | People without a big GPU | Practical speeds, since only about 3B parameters are active | ### How to set it up - Python and pip - A current llama.cpp build that supports this model's design, because older builds cannot load it - Memory for the whole file: 16 GB for the smallest, 24 GB of VRAM for Q4\_K\_M on a GPU, or 32 GB of RAM on a CPU 1. Install the Hugging Face hub tools. 2. Download the model files, or pick one file from the repository. 3. Check the files against the SHA256SUMS list, then run one with your runtime. ```sh pip install -U huggingface_hub hf download empero-ai/Qwen3.8-35B-A3B-Distill-GGUF ``` To check the downloads against the list of checksums in the repository: ```bash sha256sum -c SHA256SUMS --ignore-missing ``` ### How to use it Run the file in llama.cpp with the built-in chat template. This is a reasoning model, so every answer opens with a think block. Allow a generous token limit and strip that block before you show answers to users. Other runtimes load the file directly, and the chat template is stored inside it. ```bash llama-cli -m Qwen3.8-35B-A3B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` For images, pair the projector file with any text file. ```bash llama-mtmd-cli -m Qwen3.8-35B-A3B-Q4_K_M.gguf \ --mmproj mmproj-Qwen3.8-35B-A3B-F16.gguf \ --image photo.jpg -p "Describe this image." ``` | Tool | What it does | | --- | --- | | llama-cli | Runs a text chat with the chosen GGUF file | | llama-mtmd-cli | Runs the model on an image with the projector file | | sha256sum | Checks the downloaded files against the SHA256SUMS list | ### Example You run the Q4\_K\_M file with llama-cli and ask a question. The model writes a think block first and then its answer. For a photo, you run llama-mtmd-cli with the projector and the prompt that asks it to describe the image. ### For newcomers and experts If you are new to this, start with the Q4\_K\_M file, which the card recommends for most people. Check that your llama.cpp build is recent, then ask a short question. Inside, 30 of its layers are Gated DeltaNet and 10 use full attention. It holds 256 experts and sends each token to 8 of them. Only the whole file must fit in memory, not the 35B of compute. The IQ quants and the 2-bit and 3-bit K-quants use an importance matrix to stay coherent. The vision tower comes unchanged from the base, and the card says the distillation was text-only and vision was not evaluated. Pick a file that fits your memory → Check it against SHA256SUMS → Run it in llama.cpp or another app → Strip the think block from answers ### Good to know - Memory: the whole file must fit, and a long context adds a large cache cost - Vision: the projector is inherited from the base, and the card says vision was not evaluated - Reference file: the BF16 file is full precision and meant for reference only File size of selected quantizations in GB | Label | Value | | --- | --- | | IQ2\_M | 12.558 GB | | IQ3\_M | 16.340 GB | | Q3\_K\_M | 17.664 GB | | IQ4\_XS | 19.628 GB | | Q4\_K\_M | 21.713 GB | | Q6\_K | 29.209 GB | | Q8\_0 | 37.802 GB | | BF16 | 71.067 GB | - License: Apache-2.0 - Cost to run: no API key or paid service is named, only your own hardware - Data it reaches: downloading fetches model files, and chats run locally in your runtime - Not for: older llama.cpp builds, which fail to load this architecture ## README ````markdown # Qwen3.8-35B-A3B-Distill-GGUF Runs a 35B Mixture-of-Experts model with about 3B active parameters per token from GGUF files in ten sizes, plus a vision projector. For llama.cpp, Ollama, LM Studio, Jan and KoboldCpp users. Made by empero-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository packs empero-ai's Qwen3.8-35B-A3B-Distill into GGUF files of different sizes. That model was distilled from the Qwen3.8 frontier models into a Qwen3.6-35B-A3B mixture-of-experts network. It has 35B parameters in total, and about 3B work on each token. You can load the files in llama.cpp, in Ollama, or in desktop apps such as LM Studio, Jan and KoboldCpp. Who it suits: People who run large models on their own machine and want a file sized for their memory, including those with only a CPU and plenty of RAM. ```table Industry | Who uses it | How they use it | Example Software development | Local-model users | Run a reasoning model offline | A chat session in llama.cpp AI tooling | Desktop app users | Load the file in LM Studio, Jan or KoboldCpp | A local chat app using the Q4_K_M file Research | Model researchers | Compare quantization levels | Q2_K against Q8_0 on one model Vision work | Hobbyists | Describe images with the vision projector | A photo described through llama-mtmd-cli ``` If you have less than 16 GB of memory, you can skip it, since even the smallest file is about 12.6 GB. Where to use it: - llama.cpp, Ollama, LM Studio, Jan and KoboldCpp - Windows, macOS or Linux, after you fetch the files with Python - A machine with enough VRAM or system RAM to hold the whole file ```table Use case | Who | What they get back Reasoning chat | Local-model users | Answers that open with a think block Image description | Hobbyists | A description of a photo, using the vision projector CPU or partial offload runs | People without a big GPU | Practical speeds, since only about 3B parameters are active ``` How to set it up: - Python and pip - A current llama.cpp build that supports this model's design, because older builds cannot load it - Memory for the whole file: 16 GB for the smallest, 24 GB of VRAM for Q4_K_M on a GPU, or 32 GB of RAM on a CPU 1. Install the Hugging Face hub tools. 2. Download the model files, or pick one file from the repository. 3. Check the files against the SHA256SUMS list, then run one with your runtime. ```sh pip install -U huggingface_hub hf download empero-ai/Qwen3.8-35B-A3B-Distill-GGUF ``` To check the downloads against the list of checksums in the repository: ```bash sha256sum -c SHA256SUMS --ignore-missing ``` How to use it: Run the file in llama.cpp with the built-in chat template. This is a reasoning model, so every answer opens with a think block. Allow a generous token limit and strip that block before you show answers to users. Other runtimes load the file directly, and the chat template is stored inside it. ```bash llama-cli -m Qwen3.8-35B-A3B-Q4_K_M.gguf \ --temp 0.6 --top-p 0.95 --top-k 20 \ -n 16384 -cnv ``` For images, pair the projector file with any text file. ```bash llama-mtmd-cli -m Qwen3.8-35B-A3B-Q4_K_M.gguf \ --mmproj mmproj-Qwen3.8-35B-A3B-F16.gguf \ --image photo.jpg -p "Describe this image." ``` ```table Tool | What it does llama-cli | Runs a text chat with the chosen GGUF file llama-mtmd-cli | Runs the model on an image with the projector file sha256sum | Checks the downloaded files against the SHA256SUMS list ``` Example: You run the Q4_K_M file with llama-cli and ask a question. The model writes a think block first and then its answer. For a photo, you run llama-mtmd-cli with the projector and the prompt that asks it to describe the image. For newcomers and experts: If you are new to this, start with the Q4_K_M file, which the card recommends for most people. Check that your llama.cpp build is recent, then ask a short question. Inside, 30 of its layers are Gated DeltaNet and 10 use full attention. It holds 256 experts and sends each token to 8 of them. Only the whole file must fit in memory, not the 35B of compute. The IQ quants and the 2-bit and 3-bit K-quants use an importance matrix to stay coherent. The vision tower comes unchanged from the base, and the card says the distillation was text-only and vision was not evaluated. ```flow Pick a file that fits your memory Check it against SHA256SUMS Run it in llama.cpp or another app Strip the think block from answers ``` Good to know: - Memory: the whole file must fit, and a long context adds a large cache cost - Vision: the projector is inherited from the base, and the card says vision was not evaluated - Reference file: the BF16 file is full precision and meant for reference only ```chart Title: File size of selected quantizations in GB IQ2_M: 12.558 GB IQ3_M: 16.340 GB Q3_K_M: 17.664 GB IQ4_XS: 19.628 GB Q4_K_M: 21.713 GB Q6_K: 29.209 GB Q8_0: 37.802 GB BF16: 71.067 GB ``` - License: Apache-2.0 - Cost to run: no API key or paid service is named, only your own hardware - Data it reaches: downloading fetches model files, and chats run locally in your runtime - Not for: older llama.cpp builds, which fail to load this architecture ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 35.5B - Weights: about 17.8 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 1,337,097 - Likes: 277 - License: apache-2.0 - Languages: en - Based on: empero-ai/Qwen3.8-35B-A3B-Distill - Tags: gguf, llama.cpp, quantized, empero-ai, qwen3.6, qwen3.8, distillation, reasoning, gated-deltanet, text-generation, conversational ## Links - Model: https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF - Files: https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download empero-ai/Qwen3.8-35B-A3B-Distill-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [empero-ai/Qwen3.8-35B-A3B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-35B-A3B-Distill-GGUF (this page) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 | | [Qwen3.8-4B-Distill-GGUF](https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30) | 570,974 | 208 | 4.3B | apache-2.0 | 2026-08-16 | | [Qwen3.8-9B-Distill-GGUF](https://mindket.com/p/cfc3041d-4b8a-4012-b704-ce834019b236) | 537,695 | 356 | 9.2B | apache-2.0 | 2026-08-16 | | [Qwen3.8-2B-Distill-GGUF](https://mindket.com/p/e7e883a1-dae8-4084-aaee-c3e51741cb1b) | 509,820 | 184 | 1.9B | apache-2.0 | 2026-08-16 | | [Qwen3.8-35B-A3B-Distill](https://mindket.com/p/db646c99-e7db-4a3b-9573-75557227bfa0) | 4,390 | 103 | 35.1B | apache-2.0 | 2026-09-16 | | [Qwen3.8-27B-Ridge-GGUF](https://mindket.com/p/f254bd76-fee2-4e04-80ac-a73f9621c088) | 292,932 | 369 | 27.3B | apache-2.0 | 2026-08-15 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-09-16 - First published: 2026-09-16 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open empero-ai/Qwen3.8-35B-A3B-Distill-GGUF on Hugging Face](https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) - [Its page on Mindket](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7)