--- title: "Qwen3.5-0.8B-MTP-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/9de1e013-531a-4503-974b-0a2c0d62666f" updated: "2026-11-13" --- # Qwen3.5-0.8B-MTP-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs the small Qwen3.5 0.8B vision-language model in llama.cpp with multi-token prediction for faster generation, as Unsloth's GGUF files. For developers testing speculative decoding. Qwen3.5-0.8B-MTP-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is This repository holds Unsloth's GGUF files for Qwen3.5-0.8B with multi-token prediction, or MTP, so llama.cpp can use speculative decoding. Qwen3.5-0.8B is a small vision-language model, listed at 773M parameters. Unsloth says MTP makes generation roughly 1.5 to 2 times faster. ### Who it suits It suits developers who want to try speculative decoding on a tiny model, and people who prototype or fine-tune small models. The Qwen team says a model this small is meant for prototyping, task-specific fine-tuning and research. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Test MTP speculative decoding in llama-server | Start the card's server command and send a prompt | | Research | ML researchers | Study a small multimodal model | Compare generation speed with and without MTP | | Product teams | Prototypers | Prototype a feature on a model that is only about 386 MB at 4-bit | Run a task-specific prototype locally | Anyone who needs a large model's depth, or who needs image input through mmproj, can skip it, because MTP does not support mmproj yet. ### Where to use it - llama.cpp, built with MTP support - A CUDA GPU build, or a CPU or Metal build with CUDA switched off - Codex and OpenCode, since the card adds developer role support for them - Windows, macOS or Linux, where the files download with Python | Use case | Who | What they get back | | --- | --- | --- | | Faster local generation | Developers | Text produced about 1.5 to 2 times faster, per Unsloth | | Tool calling | Agent builders | Better parsing of nested objects in tool calls | | Local fine-tuning | Researchers | A small base to adapt with Unsloth | ### How to set it up - Python with pip, for the download - Build tools and CMake, plus the CUDA toolkit for a GPU build - A llama.cpp build that includes MTP, which the card says has been merged 1. Install the build tools. 2. Clone the llama.cpp repository. 3. Configure and build llama.cpp, then copy the binaries beside it. 4. Start llama-server with the MTP flags. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-0.8B-MTP-GGUF ``` ### How to use it Run llama-server with the card's command, which names the repository and the UD-Q4\_K\_XL tier. Set the CUDA option to OFF in the build for CPU or Metal. llama.cpp renamed the speculative type from mtp to draft-mtp, so use the new name. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-0.8B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-0.8B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` | Tool | What it does | | --- | --- | | --spec-type draft-mtp | Turns on MTP speculative decoding | | --spec-draft-n-max 6 | Sets the draft limit to 6 | | -np 1 | Keeps one slot, since values above 1 are not yet supported with MTP | | --mmproj | Not yet supported together with MTP | ### Example You start llama-server with the command above and send it a short prompt. The card says MTP speeds up generation by roughly 1.5 to 2 times. ### For newcomers and experts This is a tiny model, so it fits on modest hardware, but the setup builds llama.cpp from source. Run the build once, start the server and ask a short question. Qwen3.5 pairs Gated Delta Networks with sparse mixture-of-experts layers and is trained on multimodal tokens from the start. MTP lets the model draft several tokens ahead, which llama.cpp then checks. Build llama.cpp with MTP support → Start llama-server with draft-mtp → The model drafts several tokens → The drafts are checked and kept → Text comes out faster ### Good to know - Limits: MTP cannot yet be combined with more than one slot or with mmproj - Languages: the Qwen3.5 family covers 201 languages and dialects - Size: 773M parameters, with about 386 MB at 4-bit as an estimate - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you send to your own server, and the -hf flag pulls the files from the Hugging Face repository - Not for: batching with np above 1 or image input through mmproj, while MTP is on ## README ````markdown # Qwen3.5-0.8B-MTP-GGUF Runs the small Qwen3.5 0.8B vision-language model in llama.cpp with multi-token prediction for faster generation, as Unsloth's GGUF files. For developers testing speculative decoding. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds Unsloth's GGUF files for Qwen3.5-0.8B with multi-token prediction, or MTP, so llama.cpp can use speculative decoding. Qwen3.5-0.8B is a small vision-language model, listed at 773M parameters. Unsloth says MTP makes generation roughly 1.5 to 2 times faster. Who it suits: It suits developers who want to try speculative decoding on a tiny model, and people who prototype or fine-tune small models. The Qwen team says a model this small is meant for prototyping, task-specific fine-tuning and research. ```table Industry | Who uses it | How they use it | Example Software | Developers | Test MTP speculative decoding in llama-server | Start the card's server command and send a prompt Research | ML researchers | Study a small multimodal model | Compare generation speed with and without MTP Product teams | Prototypers | Prototype a feature on a model that is only about 386 MB at 4-bit | Run a task-specific prototype locally ``` Anyone who needs a large model's depth, or who needs image input through mmproj, can skip it, because MTP does not support mmproj yet. Where to use it: - llama.cpp, built with MTP support - A CUDA GPU build, or a CPU or Metal build with CUDA switched off - Codex and OpenCode, since the card adds developer role support for them - Windows, macOS or Linux, where the files download with Python ```table Use case | Who | What they get back Faster local generation | Developers | Text produced about 1.5 to 2 times faster, per Unsloth Tool calling | Agent builders | Better parsing of nested objects in tool calls Local fine-tuning | Researchers | A small base to adapt with Unsloth ``` How to set it up: - Python with pip, for the download - Build tools and CMake, plus the CUDA toolkit for a GPU build - A llama.cpp build that includes MTP, which the card says has been merged 1. Install the build tools. 2. Clone the llama.cpp repository. 3. Configure and build llama.cpp, then copy the binaries beside it. 4. Start llama-server with the MTP flags. ```bash apt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cpp ``` ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-0.8B-MTP-GGUF ``` How to use it: Run llama-server with the card's command, which names the repository and the UD-Q4_K_XL tier. Set the CUDA option to OFF in the build for CPU or Metal. llama.cpp renamed the speculative type from mtp to draft-mtp, so use the new name. ```bash export LLAMA_CACHE="unsloth/Qwen3.5-0.8B-MTP-GGUF" ./llama.cpp/llama-server \ -hf unsloth/Qwen3.5-0.8B-MTP-GGUF:UD-Q4_K_XL \ -ngl 99 -c 8192 -fa on -np 1 \ --spec-type draft-mtp --spec-draft-n-max 6 ``` ```table Tool | What it does --spec-type draft-mtp | Turns on MTP speculative decoding --spec-draft-n-max 6 | Sets the draft limit to 6 -np 1 | Keeps one slot, since values above 1 are not yet supported with MTP --mmproj | Not yet supported together with MTP ``` Example: You start llama-server with the command above and send it a short prompt. The card says MTP speeds up generation by roughly 1.5 to 2 times. For newcomers and experts: This is a tiny model, so it fits on modest hardware, but the setup builds llama.cpp from source. Run the build once, start the server and ask a short question. Qwen3.5 pairs Gated Delta Networks with sparse mixture-of-experts layers and is trained on multimodal tokens from the start. MTP lets the model draft several tokens ahead, which llama.cpp then checks. ```flow Build llama.cpp with MTP support Start llama-server with draft-mtp The model drafts several tokens The drafts are checked and kept Text comes out faster ``` Good to know: - Limits: MTP cannot yet be combined with more than one slot or with mmproj - Languages: the Qwen3.5 family covers 201 languages and dialects - Size: 773M parameters, with about 386 MB at 4-bit as an estimate - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you send to your own server, and the -hf flag pulls the files from the Hugging Face repository - Not for: batching with np above 1 or image input through mmproj, while MTP is on ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 773M - Weights: about 386 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 14,886 - Likes: 43 - License: apache-2.0 - Based on: Qwen/Qwen3.5-0.8B - Tags: transformers, gguf, unsloth, qwen, qwen3_5, image-text-to-text, conversational ## Links - Model: https://huggingface.co/unsloth/Qwen3.5-0.8B-MTP-GGUF - Files: https://huggingface.co/unsloth/Qwen3.5-0.8B-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.5-0.8B-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [unsloth/Qwen3.5-0.8B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-0.8B-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.5-0.8B-MTP-GGUF (this page) | 14,886 | 43 | 773M | apache-2.0 | 2026-05-16 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [Qwen3.6-27B-GGUF](https://mindket.com/p/91bfefb7-49da-47e4-a52a-f80091e27806) | 227,770 | 978 | 26.9B | apache-2.0 | 2026-04-22 | | [Qwen3.5-4B-MTP-GGUF](https://mindket.com/p/dd8b24c6-73ed-4159-b540-b7370a9077f5) | 109,500 | 112 | 4.3B | apache-2.0 | 2026-05-16 | | [Qwen3.5-9B-MTP-GGUF](https://mindket.com/p/6b8f0ca2-34dc-4ef5-b19b-4f4869d010e6) | 105,866 | 230 | 9.2B | apache-2.0 | 2026-05-16 | | [Qwen3.6-27B-MTP-GGUF](https://mindket.com/p/0818f9cc-d315-4c72-8def-581001d2b050) | 46,035 | 1,344 | 27.3B | apache-2.0 | 2026-05-26 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-05-16 - First published: 2026-05-13 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/Qwen3.5-0.8B-MTP-GGUF on Hugging Face](https://huggingface.co/unsloth/Qwen3.5-0.8B-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/9de1e013-531a-4503-974b-0a2c0d62666f) - [Its page on Mindket](https://mindket.com/p/9de1e013-531a-4503-974b-0a2c0d62666f)