--- title: "Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF)" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/5b90e607-2897-48bb-9953-ac8e99d39909" updated: "2026-11-13" --- # Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF) > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs as a local text model, a Qwen3.6-35B-A3B fine-tune trained on data written by Qwen3.8-27B and packed as one GGUF file for llama.cpp. For people who run models on their own machine for code and tool calling. Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF) is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by lododoai ## About ### What it is This is a fine-tune of Qwen3.6-35B-A3B, a mixture-of-experts model with 3B active parameters, trained on text that Qwen3.8-27B wrote as its teacher. It comes as one 22.85 GB GGUF file in the Q4\_K\_XL quant for llama.cpp. The card says it aims to match or pass the teacher while keeping the base model's speed. ### Who it suits It suits people who run language models on their own computer and care about code, tool calling and math. English and Chinese are the listed languages. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software development | Developers who want a local coding model | Run the GGUF file in llama-server and send coding prompts | A developer asks for a function and its tests | | Agent building | Teams building tool-calling agents on local hardware | Send tool-calling requests to the local server | An agent picks a tool and fills in its arguments | | Research | People who study distillation and models with few active parameters | Compare it with the base and the teacher on the card's tests | A researcher reruns a test with the card's llama.cpp settings | Anyone who cannot store a 22.85 GB file can skip the merged build and try the separate 40 MB LoRA adapter on the base model instead. ### Where to use it It runs wherever llama.cpp runs, and the file downloads on Windows, macOS or Linux with Python. - llama-server from llama.cpp: serves the model from the GGUF file - moefast: a small launcher that sets llama.cpp up for a small GPU - Strata: an engine the card names for MTP speculative decoding, which needs special sampling settings - Windows, macOS or Linux: download the files with the Hugging Face tools | Use case | Who | What they get back | | --- | --- | --- | | Local code help | Developers | Code written by a model that runs on their own machine | | Tool calling | Agent builders | A local model that chooses tools, scored 67.0% on the card's 100-question tool test | | Running on a small GPU | People with limited VRAM | A config that moefast writes next to the model, then a started server | ### How to set it up You need a few things first. - Python and the huggingface\_hub package for the download - llama.cpp with llama-server available - Memory and disk for a 22.85 GB model file - Optional: the moefast launcher for a small GPU 1. Install the Hugging Face hub package. 2. Download the model files. 3. Make sure llama-server from llama.cpp is ready to run. 4. Start llama-server with the model file, as shown under How to use it. ```sh pip install -U huggingface_hub hf download lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF ``` ### How to use it Start llama-server with the GGUF file. The card's own command sets a 32768 context and port 8080. ```sh llama-server -m Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf \ -ngl 99 --jinja -c 32768 --port 8080 ``` The card says the model has no special sampling requirement, so choose temperature and top\_p to suit your task. Each request needs the values below only if you run MTP speculative decoding on an engine, such as Strata, that adds its own sampling defaults. ```json {"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0} ``` On a small GPU, get the moefast repository from GitHub, open its folder, then run its installer and point it at your GGUF file. ```sh ./install.sh ./bin/moefast run /path/to/this-model.gguf ``` | Tool | What it does | | --- | --- | | llama-server | Serves the model from a GGUF file | | moefast run | Checks the GGUF file and your free VRAM, then writes a config you can edit beside the model and launches llama-server | | install.sh | Sets up moefast from its repository folder | | --moe-cache-mib | llama.cpp's own flag for the expert cache, which moefast enables, sizes and remembers per model | | --spec-type draft-mtp | llama.cpp's own option for MTP speculative decoding, not hit by the sampling issue | ### Example You start llama-server with the GGUF file and send the prompt "Write a Python function that merges overlapping intervals." The model's answer comes back as text from the local server. On the card's own code test of 112 AtCoder questions, this model scored 69.6%. ### For newcomers and experts This is one large file of a language model that runs on your own computer, with no cloud service named. Your first small step is to download it, start llama-server with the command above, and send it a prompt on port 8080. The base is a mixture-of-experts model with 3B active parameters. Training used QLoRA with r=16 and alpha 32 on the attention layers and shared experts, over 48.73M tokens, about 52k samples, for 0.5 epoch or 747 steps. The card averages two independent runs with equal weights to reduce random-initialisation variance, and the merged file includes the MTP head. It trades size for convenience: the merged file is 22.85 GB, while LoRA mounts are tiny but score lower on the card's code test. Qwen3.8-27B teacher writes training data → QLoRA tunes the Qwen3.6-35B-A3B base → Two runs are averaged and merged → One Q4\_K\_XL GGUF runs in llama.cpp ### Good to know - The scores are the card's own, measured with thinking disabled, a Q4 quant and the same llama.cpp flags. - On IFEval strict it scored 33.3%, below the 27B teacher's 36.7% and above the base's 24.0%. - A LoRA mount in q8\_0 is not recommended, because block quantization amplifies the error of a small delta. Use f16 for a small file. - The card says llama.cpp leaves the expert cache off by default, which roughly halves throughput. - The card calls it community work with no tie to Qwen and no Qwen endorsement. LCB AtCoder score on 112 questions | Label | Value | | --- | --- | | This model merged | 69.6% | | Qwen3.8-27B teacher | 52.7% | | Qwen3.6-35B-A3B base | 50.9% | | LoRA f16 mount | 67.9% | | LoRA f32 mount | 66.1% | | LoRA q8\_0 mount | 62.5% | - License: Apache 2.0, and the card gives the base model as Apache 2.0 too. - Cost to run: none named, no API key, account or paid service is mentioned. - Data it reaches: it downloads files from Hugging Face. llama-server runs on your machine on port 8080, and moefast reads the GGUF metadata, measures free VRAM and writes a config file next to the model. - Not for: anyone who needs an official Qwen release, since this is community work not endorsed by Qwen. ## README ````markdown # Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF) Runs as a local text model, a Qwen3.6-35B-A3B fine-tune trained on data written by Qwen3.8-27B and packed as one GGUF file for llama.cpp. For people who run models on their own machine for code and tool calling. Made by lododoai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a fine-tune of Qwen3.6-35B-A3B, a mixture-of-experts model with 3B active parameters, trained on text that Qwen3.8-27B wrote as its teacher. It comes as one 22.85 GB GGUF file in the Q4_K_XL quant for llama.cpp. The card says it aims to match or pass the teacher while keeping the base model's speed. Who it suits: It suits people who run language models on their own computer and care about code, tool calling and math. English and Chinese are the listed languages. ```table Industry | Who uses it | How they use it | Example Software development | Developers who want a local coding model | Run the GGUF file in llama-server and send coding prompts | A developer asks for a function and its tests Agent building | Teams building tool-calling agents on local hardware | Send tool-calling requests to the local server | An agent picks a tool and fills in its arguments Research | People who study distillation and models with few active parameters | Compare it with the base and the teacher on the card's tests | A researcher reruns a test with the card's llama.cpp settings ``` Anyone who cannot store a 22.85 GB file can skip the merged build and try the separate 40 MB LoRA adapter on the base model instead. Where to use it: It runs wherever llama.cpp runs, and the file downloads on Windows, macOS or Linux with Python. - llama-server from llama.cpp: serves the model from the GGUF file - moefast: a small launcher that sets llama.cpp up for a small GPU - Strata: an engine the card names for MTP speculative decoding, which needs special sampling settings - Windows, macOS or Linux: download the files with the Hugging Face tools ```table Use case | Who | What they get back Local code help | Developers | Code written by a model that runs on their own machine Tool calling | Agent builders | A local model that chooses tools, scored 67.0% on the card's 100-question tool test Running on a small GPU | People with limited VRAM | A config that moefast writes next to the model, then a started server ``` How to set it up: You need a few things first. - Python and the huggingface_hub package for the download - llama.cpp with llama-server available - Memory and disk for a 22.85 GB model file - Optional: the moefast launcher for a small GPU 1. Install the Hugging Face hub package. 2. Download the model files. 3. Make sure llama-server from llama.cpp is ready to run. 4. Start llama-server with the model file, as shown under How to use it. ```sh pip install -U huggingface_hub hf download lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF ``` How to use it: Start llama-server with the GGUF file. The card's own command sets a 32768 context and port 8080. ```sh llama-server -m Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf \ -ngl 99 --jinja -c 32768 --port 8080 ``` The card says the model has no special sampling requirement, so choose temperature and top_p to suit your task. Each request needs the values below only if you run MTP speculative decoding on an engine, such as Strata, that adds its own sampling defaults. ```json {"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0} ``` On a small GPU, get the moefast repository from GitHub, open its folder, then run its installer and point it at your GGUF file. ```sh ./install.sh ./bin/moefast run /path/to/this-model.gguf ``` ```table Tool | What it does llama-server | Serves the model from a GGUF file moefast run | Checks the GGUF file and your free VRAM, then writes a config you can edit beside the model and launches llama-server install.sh | Sets up moefast from its repository folder --moe-cache-mib | llama.cpp's own flag for the expert cache, which moefast enables, sizes and remembers per model --spec-type draft-mtp | llama.cpp's own option for MTP speculative decoding, not hit by the sampling issue ``` Example: You start llama-server with the GGUF file and send the prompt "Write a Python function that merges overlapping intervals." The model's answer comes back as text from the local server. On the card's own code test of 112 AtCoder questions, this model scored 69.6%. For newcomers and experts: This is one large file of a language model that runs on your own computer, with no cloud service named. Your first small step is to download it, start llama-server with the command above, and send it a prompt on port 8080. The base is a mixture-of-experts model with 3B active parameters. Training used QLoRA with r=16 and alpha 32 on the attention layers and shared experts, over 48.73M tokens, about 52k samples, for 0.5 epoch or 747 steps. The card averages two independent runs with equal weights to reduce random-initialisation variance, and the merged file includes the MTP head. It trades size for convenience: the merged file is 22.85 GB, while LoRA mounts are tiny but score lower on the card's code test. ```flow Qwen3.8-27B teacher writes training data QLoRA tunes the Qwen3.6-35B-A3B base Two runs are averaged and merged One Q4_K_XL GGUF runs in llama.cpp ``` Good to know: - The scores are the card's own, measured with thinking disabled, a Q4 quant and the same llama.cpp flags. - On IFEval strict it scored 33.3%, below the 27B teacher's 36.7% and above the base's 24.0%. - A LoRA mount in q8_0 is not recommended, because block quantization amplifies the error of a small delta. Use f16 for a small file. - The card says llama.cpp leaves the expert cache off by default, which roughly halves throughput. - The card calls it community work with no tie to Qwen and no Qwen endorsement. ```chart Title: LCB AtCoder score on 112 questions This model merged: 69.6% Qwen3.8-27B teacher: 52.7% Qwen3.6-35B-A3B base: 50.9% LoRA f16 mount: 67.9% LoRA f32 mount: 66.1% LoRA q8_0 mount: 62.5% ``` - License: Apache 2.0, and the card gives the base model as Apache 2.0 too. - Cost to run: none named, no API key, account or paid service is mentioned. - Data it reaches: it downloads files from Hugging Face. llama-server runs on your machine on port 8080, and moefast reads the GGUF metadata, measures free VRAM and writes a config file next to the model. - Not for: anyone who needs an official Qwen release, since this is community work not endorsed by Qwen. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: transformers - Parameters: 35.5B - Weights: about 17.8 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 746 - Likes: 8 - License: apache-2.0 - Languages: en, zh - Based on: unsloth/Qwen3.6-35B-A3B-MTP-GGUF - Tags: transformers, gguf, llama.cpp, mixture-of-experts, qwen, qwen3, qwen3.6, qwen3.8, distillation, coding, function-calling, chinese ## Links - Model: https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF - Files: https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF) (this page) | 746 | 8 | 35.5B | apache-2.0 | 2026-10-23 | | [Qwen3.8-35B-A3B-Distill](https://mindket.com/p/db646c99-e7db-4a3b-9573-75557227bfa0) | 4,390 | 103 | 35.1B | apache-2.0 | 2026-09-16 | | [Qwen3.8-35B-A3B-Distill-GGUF](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 | | [Qwen Sharp Chat Templates](https://mindket.com/p/a3a1ee62-ff71-4757-8f7c-fd63eb8a2d78) | 0 | 502 | — | apache-2.0 | 2026-10-10 | | [Qwen-Fixed-Chat-Templates](https://mindket.com/p/5d15f0b4-2436-4b6d-a715-0b40f1910b29) | 0 | 1,901 | — | apache-2.0 | 2026-09-04 | | [Qwen3.8-9B-Distill](https://mindket.com/p/f257e0b9-f595-4d2e-8a06-68cc42f90540) | 6,592 | 237 | 9.7B | apache-2.0 | 2026-08-15 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-23 - First published: 2026-10-22 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/5b90e607-2897-48bb-9953-ac8e99d39909) - [Its page on Mindket](https://mindket.com/p/5b90e607-2897-48bb-9953-ac8e99d39909)