--- title: "Qwen3.8-27B-DSpine-b16-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/9955f2b3-7961-45ce-b63c-5a324a506318" updated: "2026-11-12" --- # Qwen3.8-27B-DSpine-b16-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. A GGUF draft model that speeds up Qwen3.8-27B in llama.cpp through speculative decoding, reaching about 125 tokens per second on code and math on one RTX 3090. For people who run Qwen3.8-27B locally. Qwen3.8-27B-DSpine-b16-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by aj9o9 ## About ### What it is This repository holds GGUF conversions of DSpine, a draft model that speeds up Qwen3.8-27B through speculative decoding in llama.cpp. A small drafter proposes tokens and the large model checks them. On one RTX 3090 with a Q4\_K\_M target, the card reports about 125 tokens per second on code and math. ### Who it suits It suits people who run Qwen3.8-27B on their own NVIDIA GPU and want faster text generation, especially for code. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Speed up a local coding model | Code completion that returns sooner | | Local AI | Hobbyists with a 24 GB GPU | Get more tokens per second from the same card | Faster chat on an RTX 3090 | | AI research | Inference researchers | Study speculative decoding drafters | Compare DSpine with DFlash2 on the same target | | Infrastructure | ML engineers | Tune draft length for a given GPU | Test smaller drafts on cards where small batches are slow | If you need stock llama.cpp with no custom build, you can skip it, because upstream does not support DSpine yet. ### Where to use it - The dspine branch of a llama.cpp fork named in the card, built with CUDA - A GPU such as the RTX 3090, with 24 GB of memory - Machines where you can build llama.cpp with cmake | Use case | Who | What they get back | | --- | --- | --- | | Faster code generation | Developers | About 125 tokens per second on code and math, by the card's test | | Faster chat | Local AI users | About 90 tokens per second on a mixed set of chat tasks | | Tuning on other GPUs | Researchers | Reports from AMD, Intel and DGX Spark users that the card collects | ### How to set it up You need these first: - A target GGUF of Qwen3.8-27B, for example in Q4\_K\_M, because these files hold only the draft model - An NVIDIA GPU, since the build command turns on CUDA - Git, cmake and a C++ compiler - Python, for the Hugging Face download command 1. Download the draft model files with the hub tool. 2. Get a Qwen3.8-27B target GGUF. 3. Clone the dspine branch of the fork the card names, then build it with the commands below. 4. Start the server with the run command in the next section. ```sh pip install -U huggingface_hub hf download aj9o9/Qwen3.8-27B-DSpine-b16-GGUF ``` ```bash cd llama.cpp cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j ``` ### How to use it Start llama-server with your target model and the draft model, then chat with it. Set the draft length and the context size yourself rather than using the defaults. ```bash ./build/bin/llama-server \ -m Qwen3.8-27B-Q4_K_M.gguf \ -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \ --spec-type draft-dflash \ --spec-draft-n-max 15 \ -ngl 999 --spec-draft-ngl 999 \ -fa on --jinja -c 32768 -np 1 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 ``` | Tool | What it does | | --- | --- | | -m | Loads the Qwen3.8-27B target model | | -md | Loads the DSpine draft model | | --spec-draft-n-max | Sets how many tokens are drafted, where 15 is best on NVIDIA CUDA and the default is 3 | | -c | Sets the context size, which you should choose yourself | | --spec-draft-sampling probabilistic | Optional, and helps slightly when the temperature is above 0 | ### Example You start the server with the command above and ask for a Python function that merges two sorted lists. The target model checks each block of drafted tokens, so the answer arrives faster than without a drafter, while the target still decides what is kept. ### For newcomers and experts Speculative decoding lets a small model guess ahead so the big one can check many tokens at once. A good first step is to try the recommended Q4\_K\_M file with the settings above. DSpine extends DFlash with a five-layer drafter. It takes the hidden states of five target layers as input and drafts 16 tokens at once, as one block. It adds causal block attention, a code bridge after each layer and a refinement step that builds a 16 by 16 transition table. The draft model reads the target's hidden states → It drafts a block of 16 tokens → The target model checks the block → Accepted tokens are kept ### Good to know Speed depends on your GPU, and the numbers below come from one RTX 3090. Decode speed on one RTX 3090 in tokens per second | Label | Value | | --- | --- | | No speculative decoding | 28 tok/s | | DFlash2 | 52 tok/s | | DSpine on mixed chat | 90 tok/s | | DSpine on code and math | 125 tok/s | - Custom build: the standard llama.cpp will not open these files, so use the dspine branch of the fork - Files: Q4\_K\_M at 1.26 GB is recommended, Q8\_0 at 2.20 GB is slightly slower, and BF16 at 4.06 GB is for requantizing - Context on 24 GB: with 15 drafted tokens it fits up to a context of 65536, and with 7 it fits up to 98304 - Other GPUs: a 15-token draft may cost more time than it saves when a card handles small batches slowly - License: Apache 2.0 - Cost to run: none named, you supply the GPU - Data it reaches: the prompts you send to your own local server ## README ````markdown # Qwen3.8-27B-DSpine-b16-GGUF A GGUF draft model that speeds up Qwen3.8-27B in llama.cpp through speculative decoding, reaching about 125 tokens per second on code and math on one RTX 3090. For people who run Qwen3.8-27B locally. Made by aj9o9 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF conversions of DSpine, a draft model that speeds up Qwen3.8-27B through speculative decoding in llama.cpp. A small drafter proposes tokens and the large model checks them. On one RTX 3090 with a Q4_K_M target, the card reports about 125 tokens per second on code and math. Who it suits: It suits people who run Qwen3.8-27B on their own NVIDIA GPU and want faster text generation, especially for code. ```table Industry | Who uses it | How they use it | Example Software | Developers | Speed up a local coding model | Code completion that returns sooner Local AI | Hobbyists with a 24 GB GPU | Get more tokens per second from the same card | Faster chat on an RTX 3090 AI research | Inference researchers | Study speculative decoding drafters | Compare DSpine with DFlash2 on the same target Infrastructure | ML engineers | Tune draft length for a given GPU | Test smaller drafts on cards where small batches are slow ``` If you need stock llama.cpp with no custom build, you can skip it, because upstream does not support DSpine yet. Where to use it: - The dspine branch of a llama.cpp fork named in the card, built with CUDA - A GPU such as the RTX 3090, with 24 GB of memory - Machines where you can build llama.cpp with cmake ```table Use case | Who | What they get back Faster code generation | Developers | About 125 tokens per second on code and math, by the card's test Faster chat | Local AI users | About 90 tokens per second on a mixed set of chat tasks Tuning on other GPUs | Researchers | Reports from AMD, Intel and DGX Spark users that the card collects ``` How to set it up: You need these first: - A target GGUF of Qwen3.8-27B, for example in Q4_K_M, because these files hold only the draft model - An NVIDIA GPU, since the build command turns on CUDA - Git, cmake and a C++ compiler - Python, for the Hugging Face download command 1. Download the draft model files with the hub tool. 2. Get a Qwen3.8-27B target GGUF. 3. Clone the dspine branch of the fork the card names, then build it with the commands below. 4. Start the server with the run command in the next section. ```sh pip install -U huggingface_hub hf download aj9o9/Qwen3.8-27B-DSpine-b16-GGUF ``` ```bash cd llama.cpp cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j ``` How to use it: Start llama-server with your target model and the draft model, then chat with it. Set the draft length and the context size yourself rather than using the defaults. ```bash ./build/bin/llama-server \ -m Qwen3.8-27B-Q4_K_M.gguf \ -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \ --spec-type draft-dflash \ --spec-draft-n-max 15 \ -ngl 999 --spec-draft-ngl 999 \ -fa on --jinja -c 32768 -np 1 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --spec-draft-type-k q8_0 --spec-draft-type-v q8_0 ``` ```table Tool | What it does -m | Loads the Qwen3.8-27B target model -md | Loads the DSpine draft model --spec-draft-n-max | Sets how many tokens are drafted, where 15 is best on NVIDIA CUDA and the default is 3 -c | Sets the context size, which you should choose yourself --spec-draft-sampling probabilistic | Optional, and helps slightly when the temperature is above 0 ``` Example: You start the server with the command above and ask for a Python function that merges two sorted lists. The target model checks each block of drafted tokens, so the answer arrives faster than without a drafter, while the target still decides what is kept. For newcomers and experts: Speculative decoding lets a small model guess ahead so the big one can check many tokens at once. A good first step is to try the recommended Q4_K_M file with the settings above. DSpine extends DFlash with a five-layer drafter. It takes the hidden states of five target layers as input and drafts 16 tokens at once, as one block. It adds causal block attention, a code bridge after each layer and a refinement step that builds a 16 by 16 transition table. ```flow The draft model reads the target's hidden states It drafts a block of 16 tokens The target model checks the block Accepted tokens are kept ``` Good to know: Speed depends on your GPU, and the numbers below come from one RTX 3090. ```chart Title: Decode speed on one RTX 3090 in tokens per second No speculative decoding: 28 tok/s DFlash2: 52 tok/s DSpine on mixed chat: 90 tok/s DSpine on code and math: 125 tok/s ``` - Custom build: the standard llama.cpp will not open these files, so use the dspine branch of the fork - Files: Q4_K_M at 1.26 GB is recommended, Q8_0 at 2.20 GB is slightly slower, and BF16 at 4.06 GB is for requantizing - Context on 24 GB: with 15 drafted tokens it fits up to a context of 65536, and with 7 it fits up to 98304 - Other GPUs: a 15-token draft may cost more time than it saves when a card handles small batches slowly - License: Apache 2.0 - Cost to run: none named, you supply the GPU - Data it reaches: the prompts you send to your own local server ## On Hugging Face, 13 November 2026 - Task: text generation - Library: gguf - Parameters: 2B - Weights: about 1 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 15,022 - Likes: 41 - License: apache-2.0 - Based on: tencent/Qwen3.8-27B-DSpine-b16 - Tags: gguf, llama.cpp, dspine, dflash, speculative-decoding, draft-model, qwen3.8, text-generation, conversational ## Links - Model: https://huggingface.co/aj9o9/Qwen3.8-27B-DSpine-b16-GGUF - Files: https://huggingface.co/aj9o9/Qwen3.8-27B-DSpine-b16-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download aj9o9/Qwen3.8-27B-DSpine-b16-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [aj9o9/Qwen3.8-27B-DSpine-b16-GGUF on Hugging Face](https://huggingface.co/aj9o9/Qwen3.8-27B-DSpine-b16-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-DSpine-b16-GGUF (this page) | 15,022 | 41 | 2B | apache-2.0 | 2026-11-09 | | [DSpine for Qwen3.8-27B](https://mindket.com/p/36267b34-fc36-443e-a836-d760c0224141) | 4,332 | 222 | 2B | apache-2.0 | 2026-11-09 | | [Bielik-11B-v3.1-DFlash2](https://mindket.com/p/2711a318-550d-40de-b3db-4e250b26a909) | 44 | 3 | 1.2B | apache-2.0 | 2026-11-11 | | [Bielik-11B-v3.1-DFlash](https://mindket.com/p/2f296c12-93ae-4573-91ac-4ee308bc1fdf) | 36 | 3 | 1.2B | apache-2.0 | 2026-11-11 | | [Qwen3.8-27B-DFlash2-GGUF by z-lab](https://mindket.com/p/a7a65cb5-6162-4ab9-9b95-df77f2a95e62) | 506,737 | 162 | 1.9B | apache-2.0 | 2026-08-24 | | [Qwen3.8-Flash-Next-DFlash2-GGUF](https://mindket.com/p/f0d509d4-8e38-4f36-8052-59c52155132e) | 3,960 | 4 | 603M | apache-2.0 | 2026-10-31 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-09 - First published: 2026-11-08 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open aj9o9/Qwen3.8-27B-DSpine-b16-GGUF on Hugging Face](https://huggingface.co/aj9o9/Qwen3.8-27B-DSpine-b16-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/9955f2b3-7961-45ce-b63c-5a324a506318) - [Its page on Mindket](https://mindket.com/p/9955f2b3-7961-45ce-b63c-5a324a506318)