Qwen3.6-35B-A3B MTP GGUF

Qwen3.6-35B-A3B MTP GGUF

Runs Qwen3.6-35B-A3B from a GGUF quantization with Multi-Token Prediction for faster GPU decoding in llama.cpp. Made by ByteShape for people running local models on a graphics card.

Made by byteshape on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a GGUF quantization of Qwen3.6-35B-A3B from ByteShape, made with its ShapeLearn method, which picks a datatype per tensor to keep quality at low bit lengths. It adds Multi-Token Prediction (MTP): the model drafts several tokens per step and the base model accepts the ones that match. It targets GPU-only inference and can read images.

Who it suits: It suits people who run Qwen3.6-35B-A3B locally with llama.cpp on a graphics card and want faster decoding, especially for code, structured output and repetitive text.

Industry | Who uses it | How they use it | Example
Software development | Developers using a local coding model | Complete code with faster decoding | Code completion gains the most from MTP
Data and automation | Teams that produce structured output | Generate predictable text such as structured records | Predictable output lets more drafted tokens be accepted
Local AI hobby | Enthusiasts with one graphics card | Choose a file size that fits their card | Listed options include files of about 10 GB and 13.6 GB

You can skip it if you want CPU-only inference, since these files are tuned for GPUs.

Where to use it:

• llama.cpp, through llama-server and its web interface
• Windows, macOS or Linux, with a recent llama.cpp build
• Graphics cards such as the RTX 4090 and RTX 5090, which have benchmark plots on the author blog

Use case | Who | What they get back
Code completion | Developers | Faster decoding, since code is easy to predict
Structured output | Automation builders | Quicker generation of regular, repetitive text
Image questions | Local model users | Answers about images, since the model is vision capable

How to set it up: You need a graphics card and a llama.cpp build new enough to know MTP.

• A GPU, because the files are tuned for GPU-only use
• A recent llama.cpp build, since the draft-mtp value was added in late 2025
• Python with the Hugging Face tools, only if you want to download the files yourself

1. Run llama-server with the help flag and check that draft-mtp is listed under the spec-type option. Update llama.cpp if it is not.
2. Pick a model from the table on the card and copy its Model Tag.
3. Start llama-server with that tag and the MTP flags.
4. Wait while llama.cpp downloads the model and the vision projector on the first run.

llama-server -hf <MODEL_TAG> --mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4

The Model Tag looks like this for the first two sizes:

byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ2_S-2.25bpw
byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ3_S-3.06bpw

To fetch the files yourself with the Hugging Face tools, use these two lines:

pip install -U huggingface_hub
hf download byteshape/Qwen3.6-35B-A3B-MTP-GGUF

How to use it: Run the command and open the web interface that llama-server serves on the port you chose. Then chat or paste code as usual. The MTP head ships within the GGUF file itself, which means you do not download a draft model.

Tool | What it does
llama-server | Runs the model and serves a web interface
-hf <MODEL_TAG> | Downloads the chosen model from Hugging Face on the first run
--mmproj-auto | Loads the vision projector so the model can work with images
--spec-type draft-mtp | Turns on MTP speculative decoding with the bundled head
--spec-draft-n-max | Sets how many tokens are drafted per step

The card suggests drafting 2 to 4 tokens ahead for most workloads, and says to sweep that value for your own use.

Example: You start llama-server with the smallest listed tag, open the web interface and paste a function you want completed. Code is easy to predict, so more drafted tokens are accepted and you may see faster decoding than with the standard release.

For newcomers and experts: Think of MTP as the model guessing a few words ahead while the full model checks each guess. Right guesses save time. A first small try is the command above with the smallest model tag.

ShapeLearn learns a datatype per tensor, and the files mix KQ and IQ quantization for throughput. The MTP head is bundled in the GGUF, so draft-mtp needs no second model. The card measures quality on six benchmarks, including BFCL-V3, LiveCodeBench V6, HumanEval and GSM8K, in thinking and instruct modes. Its speed figures are effective tokens per second, which include rejected drafts.

The base model and MTP head load together
The head drafts a few tokens ahead
The base model checks each drafted token
Matching tokens are accepted at once
Decoding finishes in fewer steps

Good to know: The card lists model files by size and bits per weight. These are the first two rows.

Title: Smallest listed model files
Model ID | Bits per weight | Size
GPU-1 | 2.25 | 10 GB
GPU-2 | 3.06 | 13.6 GB

• Workload: the speedup depends on how predictable your prompts and output are, and creative generation gains less
• Your own speed: the published figures include rejected drafts, so your results can be higher or lower
• Size: 35.5B parameters, and about 17.8 GB at 4-bit, which is an estimate
• License: Apache 2.0
• Cost to run: none named, but a graphics card is needed
• Data it reaches: llama.cpp downloads the model and vision projector from Hugging Face on the first run
• Not for: CPU-only inference, since the files are tuned for GPUs

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 35.5B
• Weights: about 17.8 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 5,523
• Likes: 93
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Tags: transformers, gguf, qwen3.6, byteshape, image-text-to-text, conversational

Links

• Model: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF
• Files: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download byteshape/Qwen3.6-35B-A3B-MTP-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.