--- title: "Qwen3.5-9B-MTP-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/6b8f0ca2-34dc-4ef5-b19b-4f4869d010e6" updated: "2026-11-12" --- # Qwen3.5-9B-MTP-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. GGUF copies of Qwen3.5 9B that support multi-token-prediction speculative decoding in llama.cpp for faster generation. For people running the model locally who can build llama.cpp. Qwen3.5-9B-MTP-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is Qwen3.5-9B-MTP-GGUF is Unsloth's GGUF version of Qwen3.5-9B, an image-and-text model with 9.2B parameters. It supports MTP speculative decoding in llama.cpp, which the card says speeds up generation by about 1.5 to 2 times. The card also notes developer role support and better tool calling. ### Who it suits People who run Qwen3.5 locally with llama.cpp and are willing to build it themselves for the faster mode. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a local model behind coding tools such as Codex or OpenCode | A local endpoint for a coding client | | Research | AI hobbyists | Test speculative decoding on a 9B model | Compare speed with and without MTP | | Education | Students | Run a current open model on one GPU | A local chat with Qwen3.5 | You can skip it if you cannot build llama.cpp, or if you need several parallel requests, since the card says more than one slot is not yet supported with MTP. ### Where to use it - llama.cpp, built from the MTP branch or a release with MTP - Linux, where the card's build commands run, with options for CPU and Metal - Coding clients such as Codex and OpenCode, per the card's developer role note | Use case | Who | What they get back | | --- | --- | --- | | Faster local chat | Developers | Text generation that is about 1.5 to 2 times faster, per the card | | Tool calling | Developers | Better parsing of nested objects in tool calls | | Fine-tuning | Tinkerers | A model you can also fine-tune locally with Unsloth | ### How to set it up - A Linux machine with build tools, cmake and an NVIDIA GPU for the CUDA build - llama.cpp built from a version that has MTP - Python and pip, if you download with the hf command 1. Install the build tools. 2. Clone the llama.cpp repository, as the card shows. 3. Build with CUDA on, or set it off for CPU or Metal. 4. Start llama-server with the MTP flags. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-9B-MTP-GGUF ``` ### How to use it Start llama-server with the repository and the UD-Q4\_K\_XL quant, and the draft MTP flags. Set the CUDA option to OFF in the build for CPU or Metal. The card says more than one slot and the mmproj file are not yet supported with MTP. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-9B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-9B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` ### Example You start the server with those flags and chat with it. Generation runs faster than plain decoding, and the card gives about 1.5 to 2 times as its figure. ### For newcomers and experts If you are new, the build step is the hard part. Start with the card's guide, and keep the context at 8192 as in the example. If you know the field, MTP uses the model's own multi-token prediction head as a draft for speculative decoding. The card notes llama.cpp renamed the flag from mtp to draft-mtp in May 2026, so use the newer name. The base model uses a hybrid of Gated Delta Networks and mixture-of-experts layers. Build llama.cpp with MTP → Download the GGUF quant → Start llama-server with draft-mtp → Faster text generation ### Good to know - License: Apache 2.0 - Cost to run: your own hardware, with about 4.6 GB for a 4-bit copy by the record's estimate - Data it reaches: the files you download and the prompts you send to your local server - Not for: several parallel requests or the mmproj image file when MTP is on ## README ````markdown # Qwen3.5-9B-MTP-GGUF GGUF copies of Qwen3.5 9B that support multi-token-prediction speculative decoding in llama.cpp for faster generation. For people running the model locally who can build llama.cpp. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Qwen3.5-9B-MTP-GGUF is Unsloth's GGUF version of Qwen3.5-9B, an image-and-text model with 9.2B parameters. It supports MTP speculative decoding in llama.cpp, which the card says speeds up generation by about 1.5 to 2 times. The card also notes developer role support and better tool calling. Who it suits: People who run Qwen3.5 locally with llama.cpp and are willing to build it themselves for the faster mode. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a local model behind coding tools such as Codex or OpenCode | A local endpoint for a coding client Research | AI hobbyists | Test speculative decoding on a 9B model | Compare speed with and without MTP Education | Students | Run a current open model on one GPU | A local chat with Qwen3.5 ``` You can skip it if you cannot build llama.cpp, or if you need several parallel requests, since the card says more than one slot is not yet supported with MTP. Where to use it: - llama.cpp, built from the MTP branch or a release with MTP - Linux, where the card's build commands run, with options for CPU and Metal - Coding clients such as Codex and OpenCode, per the card's developer role note ```table Use case | Who | What they get back Faster local chat | Developers | Text generation that is about 1.5 to 2 times faster, per the card Tool calling | Developers | Better parsing of nested objects in tool calls Fine-tuning | Tinkerers | A model you can also fine-tune locally with Unsloth ``` How to set it up: - A Linux machine with build tools, cmake and an NVIDIA GPU for the CUDA build - llama.cpp built from a version that has MTP - Python and pip, if you download with the hf command 1. Install the build tools. 2. Clone the llama.cpp repository, as the card shows. 3. Build with CUDA on, or set it off for CPU or Metal. 4. Start llama-server with the MTP flags. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-9B-MTP-GGUF ``` How to use it: Start llama-server with the repository and the UD-Q4_K_XL quant, and the draft MTP flags. Set the CUDA option to OFF in the build for CPU or Metal. The card says more than one slot and the mmproj file are not yet supported with MTP. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-9B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-9B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` Example: You start the server with those flags and chat with it. Generation runs faster than plain decoding, and the card gives about 1.5 to 2 times as its figure. For newcomers and experts: If you are new, the build step is the hard part. Start with the card's guide, and keep the context at 8192 as in the example. If you know the field, MTP uses the model's own multi-token prediction head as a draft for speculative decoding. The card notes llama.cpp renamed the flag from mtp to draft-mtp in May 2026, so use the newer name. The base model uses a hybrid of Gated Delta Networks and mixture-of-experts layers. ```flow Build llama.cpp with MTP Download the GGUF quant Start llama-server with draft-mtp Faster text generation ``` Good to know: - License: Apache 2.0 - Cost to run: your own hardware, with about 4.6 GB for a 4-bit copy by the record's estimate - Data it reaches: the files you download and the prompts you send to your local server - Not for: several parallel requests or the mmproj image file when MTP is on ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 9.2B - Weights: about 4.6 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 105,866 - Likes: 230 - License: apache-2.0 - Based on: Qwen/Qwen3.5-9B - Quantized versions: 1 - Tags: transformers, gguf, unsloth, qwen, qwen3_5, image-text-to-text, conversational ## Links - Model: https://huggingface.co/unsloth/Qwen3.5-9B-MTP-GGUF - Files: https://huggingface.co/unsloth/Qwen3.5-9B-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-9B-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [unsloth/Qwen3.5-9B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-9B-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.5-9B-MTP-GGUF (this page) | 105,866 | 230 | 9.2B | apache-2.0 | 2026-05-16 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [Qwen3.6-27B-GGUF](https://mindket.com/p/91bfefb7-49da-47e4-a52a-f80091e27806) | 227,770 | 978 | 26.9B | apache-2.0 | 2026-04-22 | | [Qwen3.5-4B-MTP-GGUF](https://mindket.com/p/dd8b24c6-73ed-4159-b540-b7370a9077f5) | 109,500 | 112 | 4.3B | apache-2.0 | 2026-05-16 | | [Qwen3.6-27B-MTP-GGUF](https://mindket.com/p/0818f9cc-d315-4c72-8def-581001d2b050) | 46,035 | 1,344 | 27.3B | apache-2.0 | 2026-05-26 | | [Qwen3.5-0.8B-MTP-GGUF](https://mindket.com/p/9de1e013-531a-4503-974b-0a2c0d62666f) | 14,886 | 43 | 773M | apache-2.0 | 2026-05-16 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-05-16 - First published: 2026-05-13 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/Qwen3.5-9B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-9B-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/6b8f0ca2-34dc-4ef5-b19b-4f4869d010e6) - [Its page on Mindket](https://mindket.com/p/6b8f0ca2-34dc-4ef5-b19b-4f4869d010e6)