--- title: "Qwen3.5-4B-MTP-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/dd8b24c6-73ed-4159-b540-b7370a9077f5" updated: "2026-11-13" --- # Qwen3.5-4B-MTP-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Unsloth's GGUF files of Qwen3.5-4B with multi-token prediction, which lets llama.cpp use speculative decoding for roughly 1.5 to 2 times faster generation. For people who run a small multimodal model locally. Qwen3.5-4B-MTP-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is This repository holds Unsloth's GGUF files of Qwen3.5-4B that include multi-token prediction, or MTP. With a llama.cpp build that supports MTP, the model drafts several tokens at a time and verifies them together. The card says this makes generation about 1.5 to 2 times faster. The record lists 4.3B parameters and about 2.2 GB at 4-bit. ### Who it suits People who run a small model on their own machine and want faster text generation, and developers who use it as a local backend for coding tools. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers with a local model | Serve the model through llama-server | Point a coding tool at a local server | | Local AI | Hobbyists with limited hardware | Run a small model faster | Chat with quicker replies on one GPU | | Agents | People using Codex or OpenCode | Use the developer role the card says is now supported | Run a coding agent against a local model | You can skip it if you use a llama.cpp version without MTP support, since you would not get the speed-up. ### Where to use it - A llama.cpp build with MTP support, which the card says was merged on 16 May 2026 - Unsloth, which the card says can also fine-tune the model locally - Windows, macOS or Linux, with CUDA, CPU or Metal builds | Use case | Who | What they get back | | --- | --- | --- | | Faster local generation | Anyone running llama-server | Text replies produced with speculative decoding | | Tool calling | Agent builders | Better handling of nested objects in tool calls | | Coding agents | Developers | A model that accepts the developer role | ### How to set it up You need to build llama.cpp with MTP support and download the model through it. The card's build steps are for Linux with CUDA. Set CUDA to OFF for CPU or Metal. - Python and pip, to download files yourself - A C++ build toolchain and CMake - A GPU for the CUDA build, or use CPU or Metal 1. Clone the llama.cpp repository. 2. Build it with the commands below. 3. Start llama-server with the model and MTP settings. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` To fetch the repository files yourself, Hugging Face gives these steps: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-4B-MTP-GGUF ``` ### How to use it Start llama-server with the model from this repository and the MTP drafting options. The card notes that more than one parallel slot and the mmproj option are not yet supported with MTP. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-4B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-4B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` | Tool | What it does | | --- | --- | | llama-server | Serves the model with speculative decoding on | | --spec-type draft-mtp | Turns on MTP drafting | | --spec-draft-n-max | Sets how many tokens the draft proposes at once | | -np 1 | Keeps one parallel slot, as MTP needs | ### Example You start the server with the command above and send a chat request from your tool. The model replies with text, produced with MTP drafting. The card claims it generates faster than without MTP, though it gives no timing for this exact setup. ### For newcomers and experts If you are new, build llama.cpp as shown and start the server with the one command, then chat with a short question. The card also links an Unsloth guide for running Qwen3.5. For experts, MTP is speculative decoding where the model's own extra heads draft tokens that the main model checks together. The card notes that llama.cpp renamed the option from mtp to draft-mtp on 13 May 2026. The trade-off is that the speed-up depends on how often the drafts are accepted. Prompt in → MTP head drafts several tokens → Main model verifies them together → Accepted tokens out ### Good to know - Speed: the card claims about 1.5 to 2 times faster generation with MTP - Limits: parallel slots above one and the mmproj option are not yet supported with MTP - Quant: UD-Q4\_K\_XL is the file the sample command loads - License: Apache 2.0 - Cost to run: no API key or paid service named - Data it reaches: downloads files from Hugging Face, nothing else is named - Not for: llama.cpp builds without MTP support, or several parallel requests with MTP ## README ````markdown # Qwen3.5-4B-MTP-GGUF Unsloth's GGUF files of Qwen3.5-4B with multi-token prediction, which lets llama.cpp use speculative decoding for roughly 1.5 to 2 times faster generation. For people who run a small multimodal model locally. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds Unsloth's GGUF files of Qwen3.5-4B that include multi-token prediction, or MTP. With a llama.cpp build that supports MTP, the model drafts several tokens at a time and verifies them together. The card says this makes generation about 1.5 to 2 times faster. The record lists 4.3B parameters and about 2.2 GB at 4-bit. Who it suits: People who run a small model on their own machine and want faster text generation, and developers who use it as a local backend for coding tools. ```table Industry | Who uses it | How they use it | Example Software | Developers with a local model | Serve the model through llama-server | Point a coding tool at a local server Local AI | Hobbyists with limited hardware | Run a small model faster | Chat with quicker replies on one GPU Agents | People using Codex or OpenCode | Use the developer role the card says is now supported | Run a coding agent against a local model ``` You can skip it if you use a llama.cpp version without MTP support, since you would not get the speed-up. Where to use it: - A llama.cpp build with MTP support, which the card says was merged on 16 May 2026 - Unsloth, which the card says can also fine-tune the model locally - Windows, macOS or Linux, with CUDA, CPU or Metal builds ```table Use case | Who | What they get back Faster local generation | Anyone running llama-server | Text replies produced with speculative decoding Tool calling | Agent builders | Better handling of nested objects in tool calls Coding agents | Developers | A model that accepts the developer role ``` How to set it up: You need to build llama.cpp with MTP support and download the model through it. The card's build steps are for Linux with CUDA. Set CUDA to OFF for CPU or Metal. - Python and pip, to download files yourself - A C++ build toolchain and CMake - A GPU for the CUDA build, or use CPU or Metal 1. Clone the llama.cpp repository. 2. Build it with the commands below. 3. Start llama-server with the model and MTP settings. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` To fetch the repository files yourself, Hugging Face gives these steps: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-4B-MTP-GGUF ``` How to use it: Start llama-server with the model from this repository and the MTP drafting options. The card notes that more than one parallel slot and the mmproj option are not yet supported with MTP. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-4B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-4B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` ```table Tool | What it does llama-server | Serves the model with speculative decoding on --spec-type draft-mtp | Turns on MTP drafting --spec-draft-n-max | Sets how many tokens the draft proposes at once -np 1 | Keeps one parallel slot, as MTP needs ``` Example: You start the server with the command above and send a chat request from your tool. The model replies with text, produced with MTP drafting. The card claims it generates faster than without MTP, though it gives no timing for this exact setup. For newcomers and experts: If you are new, build llama.cpp as shown and start the server with the one command, then chat with a short question. The card also links an Unsloth guide for running Qwen3.5. For experts, MTP is speculative decoding where the model's own extra heads draft tokens that the main model checks together. The card notes that llama.cpp renamed the option from mtp to draft-mtp on 13 May 2026. The trade-off is that the speed-up depends on how often the drafts are accepted. ```flow Prompt in MTP head drafts several tokens Main model verifies them together Accepted tokens out ``` Good to know: - Speed: the card claims about 1.5 to 2 times faster generation with MTP - Limits: parallel slots above one and the mmproj option are not yet supported with MTP - Quant: UD-Q4_K_XL is the file the sample command loads - License: Apache 2.0 - Cost to run: no API key or paid service named - Data it reaches: downloads files from Hugging Face, nothing else is named - Not for: llama.cpp builds without MTP support, or several parallel requests with MTP ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 4.3B - Weights: about 2.2 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 109,500 - Likes: 112 - License: apache-2.0 - Based on: Qwen/Qwen3.5-4B - Quantized versions: 1 - Tags: transformers, gguf, unsloth, qwen, qwen3_5, image-text-to-text, conversational ## Links - Model: https://huggingface.co/unsloth/Qwen3.5-4B-MTP-GGUF - Files: https://huggingface.co/unsloth/Qwen3.5-4B-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-4B-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [unsloth/Qwen3.5-4B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-4B-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.5-4B-MTP-GGUF (this page) | 109,500 | 112 | 4.3B | apache-2.0 | 2026-05-16 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [Qwen3.6-27B-GGUF](https://mindket.com/p/91bfefb7-49da-47e4-a52a-f80091e27806) | 227,770 | 978 | 26.9B | apache-2.0 | 2026-04-22 | | [Qwen3.5-9B-MTP-GGUF](https://mindket.com/p/6b8f0ca2-34dc-4ef5-b19b-4f4869d010e6) | 105,866 | 230 | 9.2B | apache-2.0 | 2026-05-16 | | [Qwen3.6-27B-MTP-GGUF](https://mindket.com/p/0818f9cc-d315-4c72-8def-581001d2b050) | 46,035 | 1,344 | 27.3B | apache-2.0 | 2026-05-26 | | [Qwen3.5-0.8B-MTP-GGUF](https://mindket.com/p/9de1e013-531a-4503-974b-0a2c0d62666f) | 14,886 | 43 | 773M | apache-2.0 | 2026-05-16 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-05-16 - First published: 2026-05-13 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/Qwen3.5-4B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-4B-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/dd8b24c6-73ed-4159-b540-b7370a9077f5) - [Its page on Mindket](https://mindket.com/p/dd8b24c6-73ed-4159-b540-b7370a9077f5)