--- title: "Qwen3.6-35B-A3B-MTP-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/90db7fe0-7044-4238-a508-5d2ecb760999" updated: "2026-11-12" --- # Qwen3.6-35B-A3B-MTP-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Qwen3.6-35B-A3B locally from Unsloth's GGUF files with multi-token prediction for faster decoding in llama.cpp. Suits developers who want a local agentic coding model. Qwen3.6-35B-A3B-MTP-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is Unsloth's GGUF build of Qwen/Qwen3.6-35B-A3B that includes multi-token prediction, or MTP, support. Qwen3.6-35B-A3B is a 35B mixture-of-experts model with 3B active parameters, a vision encoder, and a focus on agentic coding. The card says MTP makes inference about 1.5 to 2 times faster with no accuracy loss. ### Who it suits Developers who want a local coding and agent model and care about decoding speed. It also suits people who use Unsloth Studio and want ideal MTP settings chosen for them. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a local coding model for frontend and repository work | Ask it to fix a bug across a repo | | Agents | Agent builders | Use developer-role support in coding agents | Plug it into Codex or OpenCode | | Research | Inference engineers | Test speculative decoding with MTP | Compare speed with and without MTP | If you need several parallel requests or image input with MTP, you can skip it, since the card says neither is supported yet. ### Where to use it - llama.cpp, using llama-server - Unsloth Studio on Mac, CPU or GPU - Windows, macOS or Linux | Use case | Who | What they get back | | --- | --- | --- | | Agentic coding | Developers | Frontend and repository-level reasoning | | Faster local chat | Anyone | Quicker replies through MTP drafting | | Tool calling | Agent builders | Better parsing of nested objects in tool calls | ### How to set it up - Python with pip, to download the files - A llama.cpp build with CUDA, or with CUDA off for CPU and Metal - Memory for about 17.8 GB at 4-bit by Hugging Face's estimate 1. Install the Hugging Face hub package. 2. Clone llama.cpp and build llama-server with cmake. 3. Start llama-server with the model and the MTP flags. ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF ``` The card's build lines, run after cloning llama.cpp, look like this. Set -DGGML\_CUDA=OFF for CPU or Metal. ```bash cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` ### How to use it Start llama-server with the draft-mtp speculative type. The card says np above 1 and mmproj do not work with MTP yet. ```bash export LLAMA_CACHE="unsloth/Qwen3.6-35B-A3B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 2 ``` | Tool | What it does | | --- | --- | | llama-server | Serves the model, here with MTP drafting turned on | | Unsloth Studio | Runs and fine-tunes the model, and sets MTP options for your hardware | ### Example You start llama-server with the command above and ask a coding question in the chat. The model drafts several tokens ahead and checks them, so replies come faster than without MTP. ### For newcomers and experts If you are new, Unsloth Studio is the easier route, since it picks the MTP settings for your hardware. Use the llama-server command when you want full control. Qwen3.6-35B-A3B has 40 layers in a repeating layout of Gated DeltaNet and Gated Attention blocks with mixture-of-experts feed-forward layers, 256 experts in all, with 8 routed experts and 1 shared expert active. Qwen3.6 adds an option to keep reasoning context from earlier messages. Download the MTP GGUF file → Build llama.cpp with llama-server → Start the server with draft-mtp → The model drafts and verifies tokens → Replies arrive faster ### Good to know - The speedup of about 1.5 to 2 times is the card's own claim - Parallel requests and mmproj image input are not yet supported with MTP - License: Apache 2.0 - Cost to run: none named, you supply the hardware - Data it reaches: your prompts and the files downloaded from Hugging Face - Not for: several simultaneous requests or image input while MTP is on ## README ````markdown # Qwen3.6-35B-A3B-MTP-GGUF Runs Qwen3.6-35B-A3B locally from Unsloth's GGUF files with multi-token prediction for faster decoding in llama.cpp. Suits developers who want a local agentic coding model. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Unsloth's GGUF build of Qwen/Qwen3.6-35B-A3B that includes multi-token prediction, or MTP, support. Qwen3.6-35B-A3B is a 35B mixture-of-experts model with 3B active parameters, a vision encoder, and a focus on agentic coding. The card says MTP makes inference about 1.5 to 2 times faster with no accuracy loss. Who it suits: Developers who want a local coding and agent model and care about decoding speed. It also suits people who use Unsloth Studio and want ideal MTP settings chosen for them. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a local coding model for frontend and repository work | Ask it to fix a bug across a repo Agents | Agent builders | Use developer-role support in coding agents | Plug it into Codex or OpenCode Research | Inference engineers | Test speculative decoding with MTP | Compare speed with and without MTP ``` If you need several parallel requests or image input with MTP, you can skip it, since the card says neither is supported yet. Where to use it: - llama.cpp, using llama-server - Unsloth Studio on Mac, CPU or GPU - Windows, macOS or Linux ```table Use case | Who | What they get back Agentic coding | Developers | Frontend and repository-level reasoning Faster local chat | Anyone | Quicker replies through MTP drafting Tool calling | Agent builders | Better parsing of nested objects in tool calls ``` How to set it up: - Python with pip, to download the files - A llama.cpp build with CUDA, or with CUDA off for CPU and Metal - Memory for about 17.8 GB at 4-bit by Hugging Face's estimate 1. Install the Hugging Face hub package. 2. Clone llama.cpp and build llama-server with cmake. 3. Start llama-server with the model and the MTP flags. ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF ``` The card's build lines, run after cloning llama.cpp, look like this. Set -DGGML_CUDA=OFF for CPU or Metal. ```bash cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` How to use it: Start llama-server with the draft-mtp speculative type. The card says np above 1 and mmproj do not work with MTP yet. ```bash export LLAMA_CACHE="unsloth/Qwen3.6-35B-A3B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 2 ``` ```table Tool | What it does llama-server | Serves the model, here with MTP drafting turned on Unsloth Studio | Runs and fine-tunes the model, and sets MTP options for your hardware ``` Example: You start llama-server with the command above and ask a coding question in the chat. The model drafts several tokens ahead and checks them, so replies come faster than without MTP. For newcomers and experts: If you are new, Unsloth Studio is the easier route, since it picks the MTP settings for your hardware. Use the llama-server command when you want full control. Qwen3.6-35B-A3B has 40 layers in a repeating layout of Gated DeltaNet and Gated Attention blocks with mixture-of-experts feed-forward layers, 256 experts in all, with 8 routed experts and 1 shared expert active. Qwen3.6 adds an option to keep reasoning context from earlier messages. ```flow Download the MTP GGUF file Build llama.cpp with llama-server Start the server with draft-mtp The model drafts and verifies tokens Replies arrive faster ``` Good to know: - The speedup of about 1.5 to 2 times is the card's own claim - Parallel requests and mmproj image input are not yet supported with MTP - License: Apache 2.0 - Cost to run: none named, you supply the hardware - Data it reaches: your prompts and the files downloaded from Hugging Face - Not for: several simultaneous requests or image input while MTP is on ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 35.5B - Weights: about 17.8 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 2,315,167 - Likes: 1,051 - License: apache-2.0 - Based on: Qwen/Qwen3.6-35B-A3B - Quantized versions: 8 - Tags: transformers, gguf, unsloth, qwen, qwen3_5_moe, image-text-to-text, imatrix, conversational ## Links - Model: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF - Files: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [unsloth/Qwen3.6-35B-A3B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.6-35B-A3B-MTP-GGUF (this page) | 2,315,167 | 1,051 | 35.5B | apache-2.0 | 2026-05-20 | | [Qwen3.6-35B-A3B-GGUF](https://mindket.com/p/9ac02ca9-6ba3-4ea0-b3f6-dcaea6e7789f) | 1,649,871 | 1,789 | 34.7B | apache-2.0 | 2026-04-20 | | [Qwen3.6-27B-GGUF](https://mindket.com/p/91bfefb7-49da-47e4-a52a-f80091e27806) | 227,770 | 978 | 26.9B | apache-2.0 | 2026-04-22 | | [Qwen3.6-27B-MTP-GGUF](https://mindket.com/p/0818f9cc-d315-4c72-8def-581001d2b050) | 46,035 | 1,344 | 27.3B | apache-2.0 | 2026-05-26 | | [Qwen3.5-35B-A3B-GGUF](https://mindket.com/p/2d6533e8-ae01-45f9-95a1-391bf062c841) | 314,679 | 885 | 34.7B | apache-2.0 | 2026-03-05 | | [Qwen3.6-35B-A3B-NVFP4-Fast](https://mindket.com/p/dab11469-0f18-43a7-b023-f4a9bfcdea9c) | 695,020 | 134 | 19.8B | apache-2.0 | 2026-07-12 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · fhars · 2026-07-29](https://news.ycombinator.com/item?id=49101266): Self-hosting Kimi K3: 20% more hardware cost, 20% better task resolution: The Qwen-35B-A3B numbers are even weirder, did you drop a digit? I get half of that speed on a AMD Radeon RX 5500 XT (RADV NAVI14) (8192 MiB) (unsloth/Qwen3.6-35B-A3B-MTP-GGUF:Q6\_K with q8\_0 context). - [Bluesky · Paolo Barbolini · 2026-05-22](https://bsky.app/profile/paolobarbolini.it/post/3mmgfxwbnik2e): Absurd results with unsloth/Qwen3.6-35B-A3B-MTP-GGUF Q4\_K\_M - [Hacker News · i\_am\_rocoe · 2026-06-25](https://news.ycombinator.com/item?id=48673871): As; HN: I was curious why MTP affects PP TPS in llama.cpp. My PoC recovers it?: If you want to run the PoC locally to replicate: Clone the masked-nextn-skip-catchup branch: https://codeberg.org/rocoe/llama.cpp/src/branch/masked-nextn... Run llama-server with at least --n-gpu-layers 99 --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 --no-cache-prompt. I used… - [Bluesky · デイリーHuggingFaceトレンド · 2026-05-19](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mm7fsf4avv25): 今日のHuggingFaceトレンド unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3Bモデルの重みと設定ファイルを配布するためのリポジトリです。 Hugging Face Transformers形式で提供されており、MTPによる高速な推論や、llama.cppおよびUnsloth Studioでの利用をサポートしています。 - [Hacker News · waezel · 2026-06-23](https://news.ycombinator.com/item?id=48645075): VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO: Crush: https://github.com/charmbracelet/crush/ The Q8\_K\_XL MTP model from Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF - [Bluesky · デイリーHuggingFaceトレンド · 2026-05-23](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mmjdcrrsyg2b): 今日のHuggingFaceトレンド unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3Bのポストトレーニング済みモデルの重みと設定ファイルを提供するためのリポジトリです。 Hugging Face Transformers形式で提供され、vLLMやSGLangなどのライブラリと互換性があります。 安定性と実用性を重視し、開発者に生産性の高いコーディング体験を提供することを目的としています。 - [Hacker News · klardotsh · 2026-06-16](https://news.ycombinator.com/item?id=48548810): Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?: 4-5 bit quants would probably fit pretty well on your rig. Check HuggingFace for Qwen3.6-35B-A3B-MTP-GGUF 1. They've also got a cool UI thing these days to help indicate which quants of a model will run on your hardware. Full octane isn't gonna fit on much of anything south of a 128GB machine… - [Bluesky · デイリーHuggingFaceトレンド · 2026-05-21](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mmeh7vcroq25): 今日のHuggingFaceトレンド unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3Bのモデル重みと設定ファイルを提供するためのリポジトリです。 安定性と実用性を重視し、特にコーディング体験の向上を目指した画像・テキスト入力対応モデルです。 Hugging Face TransformersやvLLMなどのフレームワークと互換性があり、MTPの活用により精度を維持したまま推論速度を高速化できます。 - [Hacker News · twothreeone · 2026-06-15](https://news.ycombinator.com/item?id=48545399): Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?: \> unsloth/Qwen3.6-35B-A3B-MTP-GGUF I've actually tried this exact same model locally as well.. albeit on just a single 3090 at 128k context and I got around 40-60tok/s with Q4\_K quantization. The thing that bugged me the most was really the quality of the output on moderately complex real-world… - [Bluesky · デイリーHuggingFaceトレンド · 2026-05-17](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mm2bz3w5c725): 今日のHuggingFaceトレンド unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3Bモデルの重みと設定ファイルをHugging Face Transformers形式で提供するリポジトリです。 MTP投機的デコーディングによる高速生成やツール呼び出しの改善、開発者ロールへの対応を目的としており、llama.cppやUnsloth Studioでの実行および微調整が可能です。 - [Hacker News · horsawlarway · 2026-06-15](https://news.ycombinator.com/item?id=48544680): Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?: For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with… - [Bluesky · デイリーHuggingFaceトレンド · 2026-05-15](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mlvbk6acrw22): 今日のHuggingFaceトレンド unsloth/Qwen3.6-35B-A3B-MTP-GGUF Qwen3.6-35B-A3Bモデルの高速な生成を実現するためのリポジトリです。 MTP投機的デコードを利用して生成速度を1.5倍から2倍に向上させる手法を提示しています。 llama.cppを用いたGGUF形式のモデル実行手順に加え、ツール呼び出しの改善や開発者ロールのサポートを提供しています。 ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-05-20 - First published: 2026-05-11 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/Qwen3.6-35B-A3B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/90db7fe0-7044-4238-a508-5d2ecb760999) - [Its page on Mindket](https://mindket.com/p/90db7fe0-7044-4238-a508-5d2ecb760999)