--- title: "Qwen3.8-27B iMatrix NVFP4 MTP GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/d44ebd5f-22b5-4277-81d0-197775776c4f" updated: "2026-11-13" --- # Qwen3.8-27B iMatrix NVFP4 MTP GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. A 17.1 GB mixed-precision GGUF of Qwen3.8-27B that keeps its multi-token prediction layer, tuned for 256K context with vision on a 24 GB GPU. For people who run big models locally in llama.cpp. Qwen3.8-27B iMatrix NVFP4 MTP GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by cdiamond ## About ### What it is This GGUF file of Qwen/Qwen3.8-27B is 17.1 GB and averages about 5.01 bits per weight, mixing several precisions. Most of it uses the NVFP4 format, and the author spends extra bits on attention, Gated DeltaNet and late feed-forward tensors. The trained multi-token prediction layer stays inside the same file. It is not a fine-tune. ### Who it suits It suits people who run a 27B model on a single 24 GB GPU and want a long context. It also suits those who want an optional vision path through a separate projector file. Who uses it | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Serve a 27B model locally with a long context | Run a long coding session in llama.cpp | | AI tooling | Agent builders | Use faster generation through the embedded MTP layer | Run tool-calling loops at a higher token rate | | Research | Quantization researchers | Study a mixed NVFP4 recipe | Compare perplexity against a Q4\_1 file | People without a GPU of about 24 GB can skip it, since the numbers on the card come from that class of card. ### Where to use it - llama.cpp, as the card's runtime - NVIDIA GPUs, tested on a 24 GB Blackwell workstation card - An optional second GPU for the vision projector - Windows, macOS or Linux, to download the files Top jobs | Use case | Who | What they get back | | --- | --- | --- | | Long-context chat | Developers | A 256K window that held 261,500 input tokens in the card's test | | Faster generation | Agent builders | Speed-up from the embedded MTP layer | | Image questions | Developers | Vision answers through the optional projector file | ### How to set it up - Python on your computer, for the download - The huggingface\_hub package - llama.cpp, with MTP support for the speed results - A GPU with about 24 GB for the full context 1. Install the Hugging Face hub package. 2. Download the files to your computer. 3. Load the main GGUF in llama.cpp. Add the vision projector file only if you want image input. ```sh pip install -U huggingface_hub hf download cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF ``` Files in the repository | File | Purpose | | --- | --- | | Qwen3.8-27B-iMatrix-NVFP4-MTP.gguf | The main model with one embedded MTP layer | | mmproj-Qwen3.8-27B-F16.gguf | A vision projector you add for images, kept as in the original conversion | | recipe/tensor-types.txt | The tensor overrides used for the final mix | | SHA256SUMS | Release checksums | ### How to use it The visible card text gives no run command. Point llama.cpp at the main GGUF, and add the projector file for images. The best speeds on the card came from a custom llama.cpp build, so a stock build may be slower. ### Example You load the main file on a 24 GB card and start a chat with a long prompt. The card's test filled the cache with 261,500 input tokens, generated 256 more, and finished with no truncation and no out-of-memory error. ### For newcomers and experts If you are new, quantizing means storing a model in fewer bits so it fits on your GPU. A first try is the main file alone at a normal context length, before adding vision. For experts, the base file is mostly NVFP4. Attention matrices in the 16 full-attention layers use Q5\_K, token embeddings use Q6\_K and the output head uses Q8\_0. The importance matrix came from 5,472 messages in 296 real sessions and is not distributed. Perplexity on the short control was 6.1197, which the author treats as tied with Q4\_1 at 6.1127. Download the main GGUF → Load it in llama.cpp → Add the projector for images → Chat with a long context ### Good to know - Speed: the card reports 50.441 tok/s as the mean of 10 production runs, and 59.456 tok/s with the embedded MTP layer. - Separate tests: the author keeps rows apart on purpose, so do not combine best cases. - Custom runtime: the highest figure used six unmerged llama.cpp pull requests. - License: Apache 2.0. - Cost to run: none named, and a 24 GB GPU is the card's test hardware. - Data it reaches: it downloads files from Hugging Face, and the source names nothing else. - Not for: small GPUs, or anyone who expects a quality win over Q4\_1. Generation speed on the card's test GPU | Label | Value | | --- | --- | | Target-only greedy | 21.189 tok/s | | Embedded MTP | 59.456 tok/s | ## README ````markdown # Qwen3.8-27B iMatrix NVFP4 MTP GGUF A 17.1 GB mixed-precision GGUF of Qwen3.8-27B that keeps its multi-token prediction layer, tuned for 256K context with vision on a 24 GB GPU. For people who run big models locally in llama.cpp. Made by cdiamond on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This GGUF file of Qwen/Qwen3.8-27B is 17.1 GB and averages about 5.01 bits per weight, mixing several precisions. Most of it uses the NVFP4 format, and the author spends extra bits on attention, Gated DeltaNet and late feed-forward tensors. The trained multi-token prediction layer stays inside the same file. It is not a fine-tune. Who it suits: It suits people who run a 27B model on a single 24 GB GPU and want a long context. It also suits those who want an optional vision path through a separate projector file. ```table Title: Who uses it Industry | Who uses it | How they use it | Example Software | Developers | Serve a 27B model locally with a long context | Run a long coding session in llama.cpp AI tooling | Agent builders | Use faster generation through the embedded MTP layer | Run tool-calling loops at a higher token rate Research | Quantization researchers | Study a mixed NVFP4 recipe | Compare perplexity against a Q4_1 file ``` People without a GPU of about 24 GB can skip it, since the numbers on the card come from that class of card. Where to use it: - llama.cpp, as the card's runtime - NVIDIA GPUs, tested on a 24 GB Blackwell workstation card - An optional second GPU for the vision projector - Windows, macOS or Linux, to download the files ```table Title: Top jobs Use case | Who | What they get back Long-context chat | Developers | A 256K window that held 261,500 input tokens in the card's test Faster generation | Agent builders | Speed-up from the embedded MTP layer Image questions | Developers | Vision answers through the optional projector file ``` How to set it up: - Python on your computer, for the download - The huggingface_hub package - llama.cpp, with MTP support for the speed results - A GPU with about 24 GB for the full context 1. Install the Hugging Face hub package. 2. Download the files to your computer. 3. Load the main GGUF in llama.cpp. Add the vision projector file only if you want image input. ```sh pip install -U huggingface_hub hf download cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF ``` ```table Title: Files in the repository File | Purpose Qwen3.8-27B-iMatrix-NVFP4-MTP.gguf | The main model with one embedded MTP layer mmproj-Qwen3.8-27B-F16.gguf | A vision projector you add for images, kept as in the original conversion recipe/tensor-types.txt | The tensor overrides used for the final mix SHA256SUMS | Release checksums ``` How to use it: The visible card text gives no run command. Point llama.cpp at the main GGUF, and add the projector file for images. The best speeds on the card came from a custom llama.cpp build, so a stock build may be slower. Example: You load the main file on a 24 GB card and start a chat with a long prompt. The card's test filled the cache with 261,500 input tokens, generated 256 more, and finished with no truncation and no out-of-memory error. For newcomers and experts: If you are new, quantizing means storing a model in fewer bits so it fits on your GPU. A first try is the main file alone at a normal context length, before adding vision. For experts, the base file is mostly NVFP4. Attention matrices in the 16 full-attention layers use Q5_K, token embeddings use Q6_K and the output head uses Q8_0. The importance matrix came from 5,472 messages in 296 real sessions and is not distributed. Perplexity on the short control was 6.1197, which the author treats as tied with Q4_1 at 6.1127. ```flow Download the main GGUF Load it in llama.cpp Add the projector for images Chat with a long context ``` Good to know: - Speed: the card reports 50.441 tok/s as the mean of 10 production runs, and 59.456 tok/s with the embedded MTP layer. - Separate tests: the author keeps rows apart on purpose, so do not combine best cases. - Custom runtime: the highest figure used six unmerged llama.cpp pull requests. - License: Apache 2.0. - Cost to run: none named, and a 24 GB GPU is the card's test hardware. - Data it reaches: it downloads files from Hugging Face, and the source names nothing else. - Not for: small GPUs, or anyone who expects a quality win over Q4_1. ```chart Title: Generation speed on the card's test GPU Target-only greedy: 21.189 tok/s Embedded MTP: 59.456 tok/s ``` ## On Hugging Face, 13 November 2026 - Task: image text to text - Parameters: 461M - Weights: about 230 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 3,271,044 - Likes: 51 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Tags: gguf, qwen, qwen3.8, nvfp4, imatrix, llama.cpp, multimodal, 256k-context, image-text-to-text ## Links - Model: https://huggingface.co/cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF - Files: https://huggingface.co/cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF on Hugging Face](https://huggingface.co/cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B iMatrix NVFP4 MTP GGUF (this page) | 3,271,044 | 51 | 461M | apache-2.0 | 2026-08-17 | | [Qwen3.8-27B-Ridge-GGUF](https://mindket.com/p/f254bd76-fee2-4e04-80ac-a73f9621c088) | 292,932 | 369 | 27.3B | apache-2.0 | 2026-08-15 | | [Qwopus3.8-27B-Flash-GGUF](https://mindket.com/p/509fda01-c99c-450a-8d3e-2444ec0c6af7) | 2,153 | 248 | 461M | apache-2.0 | 2026-11-07 | | [MiniCPM-V-4.7-1B-gguf](https://mindket.com/p/f793ef5c-72e1-400b-91fa-5b2c31b01f3f) | 168,636 | 114 | 1B | apache-2.0 | 2026-11-11 | | [Qwen3.8-27B-NVFP4-DGX-SPARK](https://mindket.com/p/a7666325-428a-4e43-b11c-5fec915dc5df) | 41,306 | 1,127 | 18.2B | apache-2.0 | 2026-10-28 | | [Qwen3.8-Flash-Coder APEX-I-MiniPlus V2.1 GGUF](https://mindket.com/p/238d4a60-b5ce-4e02-a828-5e9dce37a102) | 1,585 | 8 | 42.6B | apache-2.0 | 2026-10-11 | Numbers as Mindket last read them, on 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-08-17 - First published: 2026-08-17 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF on Hugging Face](https://huggingface.co/cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/d44ebd5f-22b5-4277-81d0-197775776c4f) - [Its page on Mindket](https://mindket.com/p/d44ebd5f-22b5-4277-81d0-197775776c4f)