Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF)

Qwen3.6-35B-A3B lododo-Distill-v1.0 (GGUF)

Runs as a local text model, a Qwen3.6-35B-A3B fine-tune trained on data written by Qwen3.8-27B and packed as one GGUF file for llama.cpp. For people who run models on their own machine for code and tool calling.

Made by lododoai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a fine-tune of Qwen3.6-35B-A3B, a mixture-of-experts model with 3B active parameters, trained on text that Qwen3.8-27B wrote as its teacher. It comes as one 22.85 GB GGUF file in the Q4_K_XL quant for llama.cpp. The card says it aims to match or pass the teacher while keeping the base model's speed.

Who it suits: It suits people who run language models on their own computer and care about code, tool calling and math. English and Chinese are the listed languages.

Industry | Who uses it | How they use it | Example
Software development | Developers who want a local coding model | Run the GGUF file in llama-server and send coding prompts | A developer asks for a function and its tests
Agent building | Teams building tool-calling agents on local hardware | Send tool-calling requests to the local server | An agent picks a tool and fills in its arguments
Research | People who study distillation and models with few active parameters | Compare it with the base and the teacher on the card's tests | A researcher reruns a test with the card's llama.cpp settings

Anyone who cannot store a 22.85 GB file can skip the merged build and try the separate 40 MB LoRA adapter on the base model instead.

Where to use it: It runs wherever llama.cpp runs, and the file downloads on Windows, macOS or Linux with Python.

• llama-server from llama.cpp: serves the model from the GGUF file
• moefast: a small launcher that sets llama.cpp up for a small GPU
• Strata: an engine the card names for MTP speculative decoding, which needs special sampling settings
• Windows, macOS or Linux: download the files with the Hugging Face tools

Use case | Who | What they get back
Local code help | Developers | Code written by a model that runs on their own machine
Tool calling | Agent builders | A local model that chooses tools, scored 67.0% on the card's 100-question tool test
Running on a small GPU | People with limited VRAM | A config that moefast writes next to the model, then a started server

How to set it up: You need a few things first.

• Python and the huggingface_hub package for the download
• llama.cpp with llama-server available
• Memory and disk for a 22.85 GB model file
• Optional: the moefast launcher for a small GPU

1. Install the Hugging Face hub package.
2. Download the model files.
3. Make sure llama-server from llama.cpp is ready to run.
4. Start llama-server with the model file, as shown under How to use it.

pip install -U huggingface_hub
hf download lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF

How to use it: Start llama-server with the GGUF file. The card's own command sets a 32768 context and port 8080.

llama-server -m Qwen3.6-35B-A3B-lododo-Distill-v1.0-UD-Q4_K_XL.gguf \
  -ngl 99 --jinja -c 32768 --port 8080

The card says the model has no special sampling requirement, so choose temperature and top_p to suit your task. Each request needs the values below only if you run MTP speculative decoding on an engine, such as Strata, that adds its own sampling defaults.

{"temperature": 0, "presence_penalty": 0, "frequency_penalty": 0, "repeat_penalty": 1.0}

On a small GPU, get the moefast repository from GitHub, open its folder, then run its installer and point it at your GGUF file.

./install.sh
./bin/moefast run /path/to/this-model.gguf

Tool | What it does
llama-server | Serves the model from a GGUF file
moefast run | Checks the GGUF file and your free VRAM, then writes a config you can edit beside the model and launches llama-server
install.sh | Sets up moefast from its repository folder
--moe-cache-mib | llama.cpp's own flag for the expert cache, which moefast enables, sizes and remembers per model
--spec-type draft-mtp | llama.cpp's own option for MTP speculative decoding, not hit by the sampling issue

Example: You start llama-server with the GGUF file and send the prompt "Write a Python function that merges overlapping intervals." The model's answer comes back as text from the local server. On the card's own code test of 112 AtCoder questions, this model scored 69.6%.

For newcomers and experts: This is one large file of a language model that runs on your own computer, with no cloud service named. Your first small step is to download it, start llama-server with the command above, and send it a prompt on port 8080.

The base is a mixture-of-experts model with 3B active parameters. Training used QLoRA with r=16 and alpha 32 on the attention layers and shared experts, over 48.73M tokens, about 52k samples, for 0.5 epoch or 747 steps. The card averages two independent runs with equal weights to reduce random-initialisation variance, and the merged file includes the MTP head. It trades size for convenience: the merged file is 22.85 GB, while LoRA mounts are tiny but score lower on the card's code test.

Qwen3.8-27B teacher writes training data
QLoRA tunes the Qwen3.6-35B-A3B base
Two runs are averaged and merged
One Q4_K_XL GGUF runs in llama.cpp

Good to know:

• The scores are the card's own, measured with thinking disabled, a Q4 quant and the same llama.cpp flags.
• On IFEval strict it scored 33.3%, below the 27B teacher's 36.7% and above the base's 24.0%.
• A LoRA mount in q8_0 is not recommended, because block quantization amplifies the error of a small delta. Use f16 for a small file.
• The card says llama.cpp leaves the expert cache off by default, which roughly halves throughput.
• The card calls it community work with no tie to Qwen and no Qwen endorsement.

Title: LCB AtCoder score on 112 questions
This model merged: 69.6%
Qwen3.8-27B teacher: 52.7%
Qwen3.6-35B-A3B base: 50.9%
LoRA f16 mount: 67.9%
LoRA f32 mount: 66.1%
LoRA q8_0 mount: 62.5%

• License: Apache 2.0, and the card gives the base model as Apache 2.0 too.
• Cost to run: none named, no API key, account or paid service is mentioned.
• Data it reaches: it downloads files from Hugging Face. llama-server runs on your machine on port 8080, and moefast reads the GGUF metadata, measures free VRAM and writes a config file next to the model.
• Not for: anyone who needs an official Qwen release, since this is community work not endorsed by Qwen.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 35.5B
• Weights: about 17.8 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 746
• Likes: 8
• License: apache-2.0
• Languages: en, zh
• Based on: unsloth/Qwen3.6-35B-A3B-MTP-GGUF
• Tags: transformers, gguf, llama.cpp, mixture-of-experts, qwen, qwen3, qwen3.6, qwen3.8, distillation, coding, function-calling, chinese

Links

• Model: https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF
• Files: https://huggingface.co/lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download lododoai/Qwen3.6-35B-A3B-Distill-Qwen3.8-27B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.