Qwen3.6-35B-A3B-MTP-GGUF

Qwen3.6-35B-A3B-MTP-GGUF

Runs Qwen3.6-35B-A3B locally from Unsloth's GGUF files with multi-token prediction for faster decoding in llama.cpp. Suits developers who want a local agentic coding model.

Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Unsloth's GGUF build of Qwen/Qwen3.6-35B-A3B that includes multi-token prediction, or MTP, support. Qwen3.6-35B-A3B is a 35B mixture-of-experts model with 3B active parameters, a vision encoder, and a focus on agentic coding. The card says MTP makes inference about 1.5 to 2 times faster with no accuracy loss.

Who it suits: Developers who want a local coding and agent model and care about decoding speed. It also suits people who use Unsloth Studio and want ideal MTP settings chosen for them.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local coding model for frontend and repository work | Ask it to fix a bug across a repo
Agents | Agent builders | Use developer-role support in coding agents | Plug it into Codex or OpenCode
Research | Inference engineers | Test speculative decoding with MTP | Compare speed with and without MTP

If you need several parallel requests or image input with MTP, you can skip it, since the card says neither is supported yet.

Where to use it:

• llama.cpp, using llama-server
• Unsloth Studio on Mac, CPU or GPU
• Windows, macOS or Linux

Use case | Who | What they get back
Agentic coding | Developers | Frontend and repository-level reasoning
Faster local chat | Anyone | Quicker replies through MTP drafting
Tool calling | Agent builders | Better parsing of nested objects in tool calls

How to set it up:

• Python with pip, to download the files
• A llama.cpp build with CUDA, or with CUDA off for CPU and Metal
• Memory for about 17.8 GB at 4-bit by Hugging Face's estimate

1. Install the Hugging Face hub package.
2. Clone llama.cpp and build llama-server with cmake.
3. Start llama-server with the model and the MTP flags.

pip install -U huggingface_hub
hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF

The card's build lines, run after cloning llama.cpp, look like this. Set -DGGML_CUDA=OFF for CPU or Metal.

cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

How to use it: Start llama-server with the draft-mtp speculative type. The card says np above 1 and mmproj do not work with MTP yet.

export LLAMA_CACHE="unsloth/Qwen3.6-35B-A3B-MTP-GGUF"
./llama.cpp/llama-server \
    -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \
    -ngl 99 -c 8192 -fa on -np 1 \
    --spec-type draft-mtp --spec-draft-n-max 2

Tool | What it does
llama-server | Serves the model, here with MTP drafting turned on
Unsloth Studio | Runs and fine-tunes the model, and sets MTP options for your hardware

Example: You start llama-server with the command above and ask a coding question in the chat. The model drafts several tokens ahead and checks them, so replies come faster than without MTP.

For newcomers and experts: If you are new, Unsloth Studio is the easier route, since it picks the MTP settings for your hardware. Use the llama-server command when you want full control.

Qwen3.6-35B-A3B has 40 layers in a repeating layout of Gated DeltaNet and Gated Attention blocks with mixture-of-experts feed-forward layers, 256 experts in all, with 8 routed experts and 1 shared expert active. Qwen3.6 adds an option to keep reasoning context from earlier messages.

Download the MTP GGUF file
Build llama.cpp with llama-server
Start the server with draft-mtp
The model drafts and verifies tokens
Replies arrive faster

Good to know:

• The speedup of about 1.5 to 2 times is the card's own claim
• Parallel requests and mmproj image input are not yet supported with MTP
• License: Apache 2.0
• Cost to run: none named, you supply the hardware
• Data it reaches: your prompts and the files downloaded from Hugging Face
• Not for: several simultaneous requests or image input while MTP is on

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 35.5B
• Weights: about 17.8 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 2,315,167
• Likes: 1,051
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Quantized versions: 8
• Tags: transformers, gguf, unsloth, qwen, qwen3_5_moe, image-text-to-text, imatrix, conversational

Links

• Model: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF
• Files: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.