--- title: "Qwen3.6-35B-A3B MTP GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/b0e6577e-0f5b-4b48-aed6-71346c1f9043" updated: "2026-11-13" --- # Qwen3.6-35B-A3B MTP GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Qwen3.6-35B-A3B from a GGUF quantization with Multi-Token Prediction for faster GPU decoding in llama.cpp. Made by ByteShape for people running local models on a graphics card. Qwen3.6-35B-A3B MTP GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by byteshape ## About ### What it is This is a GGUF quantization of Qwen3.6-35B-A3B from ByteShape, made with its ShapeLearn method, which picks a datatype per tensor to keep quality at low bit lengths. It adds Multi-Token Prediction (MTP): the model drafts several tokens per step and the base model accepts the ones that match. It targets GPU-only inference and can read images. ### Who it suits It suits people who run Qwen3.6-35B-A3B locally with llama.cpp on a graphics card and want faster decoding, especially for code, structured output and repetitive text. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software development | Developers using a local coding model | Complete code with faster decoding | Code completion gains the most from MTP | | Data and automation | Teams that produce structured output | Generate predictable text such as structured records | Predictable output lets more drafted tokens be accepted | | Local AI hobby | Enthusiasts with one graphics card | Choose a file size that fits their card | Listed options include files of about 10 GB and 13.6 GB | You can skip it if you want CPU-only inference, since these files are tuned for GPUs. ### Where to use it - llama.cpp, through llama-server and its web interface - Windows, macOS or Linux, with a recent llama.cpp build - Graphics cards such as the RTX 4090 and RTX 5090, which have benchmark plots on the author blog | Use case | Who | What they get back | | --- | --- | --- | | Code completion | Developers | Faster decoding, since code is easy to predict | | Structured output | Automation builders | Quicker generation of regular, repetitive text | | Image questions | Local model users | Answers about images, since the model is vision capable | ### How to set it up You need a graphics card and a llama.cpp build new enough to know MTP. - A GPU, because the files are tuned for GPU-only use - A recent llama.cpp build, since the draft-mtp value was added in late 2025 - Python with the Hugging Face tools, only if you want to download the files yourself 1. Run llama-server with the help flag and check that draft-mtp is listed under the spec-type option. Update llama.cpp if it is not. 2. Pick a model from the table on the card and copy its Model Tag. 3. Start llama-server with that tag and the MTP flags. 4. Wait while llama.cpp downloads the model and the vision projector on the first run. ```sh llama-server -hf --mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4 ``` The Model Tag looks like this for the first two sizes: ```text byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ2_S-2.25bpw byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ3_S-3.06bpw ``` To fetch the files yourself with the Hugging Face tools, use these two lines: ```sh pip install -U huggingface_hub hf download byteshape/Qwen3.6-35B-A3B-MTP-GGUF ``` ### How to use it Run the command and open the web interface that llama-server serves on the port you chose. Then chat or paste code as usual. The MTP head ships within the GGUF file itself, which means you do not download a draft model. | Tool | What it does | | --- | --- | | llama-server | Runs the model and serves a web interface | | -hf \ | Downloads the chosen model from Hugging Face on the first run | | --mmproj-auto | Loads the vision projector so the model can work with images | | --spec-type draft-mtp | Turns on MTP speculative decoding with the bundled head | | --spec-draft-n-max | Sets how many tokens are drafted per step | The card suggests drafting 2 to 4 tokens ahead for most workloads, and says to sweep that value for your own use. ### Example You start llama-server with the smallest listed tag, open the web interface and paste a function you want completed. Code is easy to predict, so more drafted tokens are accepted and you may see faster decoding than with the standard release. ### For newcomers and experts Think of MTP as the model guessing a few words ahead while the full model checks each guess. Right guesses save time. A first small try is the command above with the smallest model tag. ShapeLearn learns a datatype per tensor, and the files mix KQ and IQ quantization for throughput. The MTP head is bundled in the GGUF, so draft-mtp needs no second model. The card measures quality on six benchmarks, including BFCL-V3, LiveCodeBench V6, HumanEval and GSM8K, in thinking and instruct modes. Its speed figures are effective tokens per second, which include rejected drafts. The base model and MTP head load together → The head drafts a few tokens ahead → The base model checks each drafted token → Matching tokens are accepted at once → Decoding finishes in fewer steps ### Good to know The card lists model files by size and bits per weight. These are the first two rows. Smallest listed model files | Model ID | Bits per weight | Size | | --- | --- | --- | | GPU-1 | 2.25 | 10 GB | | GPU-2 | 3.06 | 13.6 GB | - Workload: the speedup depends on how predictable your prompts and output are, and creative generation gains less - Your own speed: the published figures include rejected drafts, so your results can be higher or lower - Size: 35.5B parameters, and about 17.8 GB at 4-bit, which is an estimate - License: Apache 2.0 - Cost to run: none named, but a graphics card is needed - Data it reaches: llama.cpp downloads the model and vision projector from Hugging Face on the first run - Not for: CPU-only inference, since the files are tuned for GPUs ## README ````markdown # Qwen3.6-35B-A3B MTP GGUF Runs Qwen3.6-35B-A3B from a GGUF quantization with Multi-Token Prediction for faster GPU decoding in llama.cpp. Made by ByteShape for people running local models on a graphics card. Made by byteshape on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a GGUF quantization of Qwen3.6-35B-A3B from ByteShape, made with its ShapeLearn method, which picks a datatype per tensor to keep quality at low bit lengths. It adds Multi-Token Prediction (MTP): the model drafts several tokens per step and the base model accepts the ones that match. It targets GPU-only inference and can read images. Who it suits: It suits people who run Qwen3.6-35B-A3B locally with llama.cpp on a graphics card and want faster decoding, especially for code, structured output and repetitive text. ```table Industry | Who uses it | How they use it | Example Software development | Developers using a local coding model | Complete code with faster decoding | Code completion gains the most from MTP Data and automation | Teams that produce structured output | Generate predictable text such as structured records | Predictable output lets more drafted tokens be accepted Local AI hobby | Enthusiasts with one graphics card | Choose a file size that fits their card | Listed options include files of about 10 GB and 13.6 GB ``` You can skip it if you want CPU-only inference, since these files are tuned for GPUs. Where to use it: - llama.cpp, through llama-server and its web interface - Windows, macOS or Linux, with a recent llama.cpp build - Graphics cards such as the RTX 4090 and RTX 5090, which have benchmark plots on the author blog ```table Use case | Who | What they get back Code completion | Developers | Faster decoding, since code is easy to predict Structured output | Automation builders | Quicker generation of regular, repetitive text Image questions | Local model users | Answers about images, since the model is vision capable ``` How to set it up: You need a graphics card and a llama.cpp build new enough to know MTP. - A GPU, because the files are tuned for GPU-only use - A recent llama.cpp build, since the draft-mtp value was added in late 2025 - Python with the Hugging Face tools, only if you want to download the files yourself 1. Run llama-server with the help flag and check that draft-mtp is listed under the spec-type option. Update llama.cpp if it is not. 2. Pick a model from the table on the card and copy its Model Tag. 3. Start llama-server with that tag and the MTP flags. 4. Wait while llama.cpp downloads the model and the vision projector on the first run. ```sh llama-server -hf --mmproj-auto --spec-type draft-mtp --spec-draft-n-max 4 ``` The Model Tag looks like this for the first two sizes: ```text byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ2_S-2.25bpw byteshape/Qwen3.6-35B-A3B-MTP-GGUF:Qwen3.6-35B-A3B-IQ3_S-3.06bpw ``` To fetch the files yourself with the Hugging Face tools, use these two lines: ```sh pip install -U huggingface_hub hf download byteshape/Qwen3.6-35B-A3B-MTP-GGUF ``` How to use it: Run the command and open the web interface that llama-server serves on the port you chose. Then chat or paste code as usual. The MTP head ships within the GGUF file itself, which means you do not download a draft model. ```table Tool | What it does llama-server | Runs the model and serves a web interface -hf | Downloads the chosen model from Hugging Face on the first run --mmproj-auto | Loads the vision projector so the model can work with images --spec-type draft-mtp | Turns on MTP speculative decoding with the bundled head --spec-draft-n-max | Sets how many tokens are drafted per step ``` The card suggests drafting 2 to 4 tokens ahead for most workloads, and says to sweep that value for your own use. Example: You start llama-server with the smallest listed tag, open the web interface and paste a function you want completed. Code is easy to predict, so more drafted tokens are accepted and you may see faster decoding than with the standard release. For newcomers and experts: Think of MTP as the model guessing a few words ahead while the full model checks each guess. Right guesses save time. A first small try is the command above with the smallest model tag. ShapeLearn learns a datatype per tensor, and the files mix KQ and IQ quantization for throughput. The MTP head is bundled in the GGUF, so draft-mtp needs no second model. The card measures quality on six benchmarks, including BFCL-V3, LiveCodeBench V6, HumanEval and GSM8K, in thinking and instruct modes. Its speed figures are effective tokens per second, which include rejected drafts. ```flow The base model and MTP head load together The head drafts a few tokens ahead The base model checks each drafted token Matching tokens are accepted at once Decoding finishes in fewer steps ``` Good to know: The card lists model files by size and bits per weight. These are the first two rows. ```table Title: Smallest listed model files Model ID | Bits per weight | Size GPU-1 | 2.25 | 10 GB GPU-2 | 3.06 | 13.6 GB ``` - Workload: the speedup depends on how predictable your prompts and output are, and creative generation gains less - Your own speed: the published figures include rejected drafts, so your results can be higher or lower - Size: 35.5B parameters, and about 17.8 GB at 4-bit, which is an estimate - License: Apache 2.0 - Cost to run: none named, but a graphics card is needed - Data it reaches: llama.cpp downloads the model and vision projector from Hugging Face on the first run - Not for: CPU-only inference, since the files are tuned for GPUs ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 35.5B - Weights: about 17.8 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 5,523 - Likes: 93 - License: apache-2.0 - Based on: Qwen/Qwen3.6-35B-A3B - Tags: transformers, gguf, qwen3.6, byteshape, image-text-to-text, conversational ## Links - Model: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF - Files: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download byteshape/Qwen3.6-35B-A3B-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [byteshape/Qwen3.6-35B-A3B-MTP-GGUF on Hugging Face](https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.6-35B-A3B MTP GGUF (this page) | 5,523 | 93 | 35.5B | apache-2.0 | 2026-05-20 | | [Qwen3.6-35B-A3B GGUF (ShapeLearn Quantized)](https://mindket.com/p/d3e345c0-fa4a-456a-b684-1efcbfffbaca) | 5,517 | 49 | 34.7B | apache-2.0 | 2026-05-19 | | [Qwen3.8-27B GGUF (ShapeLearn Quantized)](https://mindket.com/p/ca45abfc-d3db-4fb5-aad9-dcf7a7a498c4) | 2,083,628 | 216 | 27.3B | apache-2.0 | 2026-09-18 | | [Qwen3.5-9B GGUF (ShapeLearn)](https://mindket.com/p/3cfc722b-f9e1-4041-bcf0-613c6ae4bcc3) | 3,023 | 41 | 9B | apache-2.0 | 2026-03-31 | | [North-Mini-Code-1.0 GGUF](https://mindket.com/p/d4ca57d1-ea08-4941-9a22-5d6b17bc3176) | 269 | 10 | 30.5B | apache-2.0 | 2026-07-06 | | [Qwen3.8-35B-A3B-Distill](https://mindket.com/p/db646c99-e7db-4a3b-9573-75557227bfa0) | 4,390 | 103 | 35.1B | apache-2.0 | 2026-09-16 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-05-20 - First published: 2026-05-19 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open byteshape/Qwen3.6-35B-A3B-MTP-GGUF on Hugging Face](https://huggingface.co/byteshape/Qwen3.6-35B-A3B-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/b0e6577e-0f5b-4b48-aed6-71346c1f9043) - [Its page on Mindket](https://mindket.com/p/b0e6577e-0f5b-4b48-aed6-71346c1f9043)