Qwen3.8-27B-DFlash2-GGUF by z-lab

Qwen3.8-27B-DFlash2-GGUF by z-lab

A draft model in GGUF form that speeds up Qwen3.8-27B by proposing token blocks for the main model to verify. For people who run that model locally with llama.cpp.

Made by z-lab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF conversions of DFlash 2, a draft model for Qwen3.8-27B. It is not a language model you can chat with. A speculative decoding server loads it next to the main model. It suggests tokens, and the main model checks each one. The repository mirrors the one published by incoai.

Who it suits: People who run Qwen3.8-27B on their own machine with llama.cpp and want faster generation are the main fit. Researchers who study speculative decoding can use it as a drafter to compare.

Industry | Who uses it | How they use it | Example
Software | Local LLM users | Pair the drafter with Qwen3.8-27B in llama.cpp | Serve the target and the draft together
AI infrastructure | Inference engineers | Test block drafting on a local server | Check acceptance length on their own prompts
Research | Decoding researchers | Study block-diffusion drafters | Compare the Q4_K_M, Q8_0 and BF16 draft files

If you do not run Qwen3.8-27B, you can skip it, because it drafts only for that target.

Where to use it: Use it with a llama.cpp build that has DFlash 2 support.

• llama.cpp built from the pull request the card names, with NVIDIA CUDA or Apple Silicon Metal
• A speculative decoding server such as llama-server
• Windows, macOS or Linux, with Python for the download

Use case | Who | What they get back
Faster local generation | Local LLM users | The same text as the target alone, drafted in blocks
Drafter comparison | Researchers | Acceptance lengths for three draft file sizes

How to set it up: You need a llama.cpp checkout and a build for your hardware.

• Git and CMake
• NVIDIA CUDA or Apple Silicon, to match the build you choose
• The Qwen3.8-27B target in GGUF form, which the serve command fetches for you
• Python with pip, if you want to download the draft files by hand

1. Clone the llama.cpp repository, as the card shows.
2. Inside it, fetch pull request 27342 into a local branch and switch to it.
3. Build with CUDA or Metal.

cd llama.cpp
git fetch origin pull/27342/head:pr-27342
git switch pr-27342

# NVIDIA CUDA
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j

# Apple Silicon
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build -j

pip install -U huggingface_hub
hf download z-lab/Qwen3.8-27B-DFlash2-GGUF

How to use it: Start llama-server with the target model and the draft model, then use the server as usual. The command below pairs the Q4_K_M files.

./build/bin/llama-server \
  -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
  -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
  --spec-type draft-dflash \
  --spec-draft-n-max 7

Tool | What it does
-hf | Names the target model that writes the final text
-hfd | Names the draft model that proposes tokens
--spec-type draft-dflash | Turns on the DFlash drafting method
--spec-draft-n-max 7 | Caps the draft length at 7 tokens

Draft file | Size
Q4_K_M | 1.1 GB
Q8_0 | 2.0 GB
BF16 | 3.8 GB

Example: You run the command above and send a chat request to the server. The draft model proposes up to 7 tokens at a time, and Qwen3.8-27B checks them before they reach your reply.

For newcomers and experts: This file is a helper, not a chat model. It makes a bigger model answer faster by guessing ahead. A good first try is the Q4_K_M pair in the command above.

DFlash 2 predicts a whole block of tokens in one pass and keeps the top candidates at each position. A small selector then picks a single consistent path through those candidates. Inside the backbone, two-tap dynamic convolutions stop the draft quality from dropping as the block goes on. The card says greedy decoding gives the same text as the target alone, and sampling keeps the target's distribution.

The draft model predicts a block in one pass
A selector picks one coherent path
Qwen3.8-27B verifies the proposed tokens
Accepted tokens join the reply

Good to know: The card reports acceptance length, which is how many tokens get through per verification step on average. A higher number is better.

Title: Acceptance length by draft file
BF16: 5.28 tokens per step
Q8_0: 5.13 tokens per step
Q4_K_M: 5.39 tokens per step

• Test size: the check used the first eight GSM8K test examples, with 2048 new tokens at most
• Sampling: temperature 1.0, top-p 0.95 and top-k 20, with xhigh reasoning effort
• Build: it needs a llama.cpp build from a pull request, not a plain release
• Versions: other people have published 1 quantized version
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: your prompts, the target model files and the draft files from Hugging Face. The card names no account or web service
• Not for: standalone chat, or any target other than Qwen3.8-27B

On Hugging Face, 14 November 2026

• Task: text generation
• Library: llama.cpp
• Parameters: 1.9B
• Weights: about 962 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 506,737
• Likes: 162
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 1
• Tags: llama.cpp, gguf, dflash2, speculative-decoding, draft-model, text-generation, conversational

Links

• Model: https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2-GGUF
• Files: https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download z-lab/Qwen3.8-27B-DFlash2-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.