Qwen3.8-Flash-Next-DFlash2-GGUF

Qwen3.8-Flash-Next-DFlash2-GGUF

GGUF draft model that speeds up Qwen3.8-Flash-Next in llama.cpp through speculative decoding, in 334 MiB and 622 MiB files. It is not a standalone model, and it suits people who run that target locally.

Made by nazeshinjite on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF conversions of DFlash 2, a small draft model for Qwen3.8-Flash-Next. It is not a standalone language model. It proposes tokens in advance, and the main Qwen3.8-Flash-Next model checks them, which speeds up generation in llama.cpp. The check is lossless, so greedy output matches the main model exactly. It has 603M parameters.

Who it suits: It suits people who already run Qwen3.8-Flash-Next with llama.cpp and want faster decoding without changing the answers.

Industry | Who uses it | How they use it | Example
Local AI | Mac users running llama.cpp | Add the drafter to speed up decoding | List and code prompts decoded faster on an Apple M5 Max
Software | Developers | Serve a local model with llama-server | Start the server with a target file and the drafter file
Research | Inference researchers | Study block-diffusion drafting | Compare it with the model's built-in draft head

If you do not run Qwen3.8-Flash-Next, you can skip it, because the drafter works only with that target.

Where to use it: It works inside llama.cpp, through its llama-server program, on a build at version 0.8.0 or newer. The card measured it on an Apple M5 Max with Metal.

• llama.cpp, version 0.8.0 or newer
• A Qwen3.8-Flash-Next GGUF file as the main model
• Windows, macOS or Linux with the huggingface_hub package, for the download

Use case | Who | What they get back
Faster code answers | Developers | 95.1 tokens per second against 80.5 with the model's own draft head
Faster list answers | Local users | 127.0 tokens per second against 91.4
Image prompts | Local users | Drafting that keeps working when you add a vision file

How to set it up: You need llama.cpp at version 0.8.0 or newer, a Qwen3.8-Flash-Next GGUF as the main model, and this drafter file. The card says earlier builds abort on the first request.

• llama.cpp, version 0.8.0 or newer
• A Qwen3.8-Flash-Next GGUF main model
• Python and the Hugging Face hub package, for the download

1. Install the Hugging Face hub package.
2. Download the drafter files.
3. Start llama-server with the main model and the drafter.

pip install -U huggingface_hub
hf download nazeshinjite/Qwen3.8-Flash-Next-DFlash2-GGUF

How to use it: Start llama-server with the main model and the drafter, and turn on DFlash drafting. The card recommends a draft length of 5 and a probability cutoff of 0.75. Add the vision file option for image prompts.

llama-server \
  -m Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf \
  -md Qwen3.8-Flash-Next-DFlash2-Q4_0.gguf \
  --spec-type draft-dflash --spec-draft-n-max 5 --spec-draft-p-min 0.75 \
  -c 131072 -np 1 -ub 1024 -ngl 99 -fa on \
  --jinja --reasoning-format deepseek \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0

Tool | What it does
-m | Names the main model file
-md | Names the drafter file
--spec-type draft-dflash | Turns on DFlash drafting
--spec-draft-n-max and --spec-draft-p-min | Set the recommended draft length of 5 and cutoff of 0.75
Q4_0 file, 334 MiB | The recommended drafter
Q8_0 file, 622 MiB | A larger drafter with the same acceptance and speed

Example: You start llama-server with the Q5 main model and the Q4_0 drafter, then ask it to write code. The card's test on an Apple M5 Max shows code decoding at 95.1 tokens per second against 80.5 with the built-in draft head. The text it writes matches the main model alone.

For newcomers and experts: If you are new to this, picture a fast helper that guesses the next words while the main model checks them. A first small step is to start llama-server with the card's command and ask for code or a list, where the card shows the biggest gains.

If you know the field, DFlash 2 is a block-diffusion drafter with 5 layers and a block size of 8. In one pass it drafts a full block, retains the strongest candidates for every slot, and a compact selector chooses a single consistent path through them. On disk it is about 6x smaller than the MTP head, but its runtime memory cost is about the same because of a 2.9 GiB compute buffer.

Drafter proposes a block of tokens
Selector picks one coherent path
Main model checks the whole path
Accepted tokens are kept as is
Output matches the main model

Good to know: On prose prompts the card shows a small slowdown, 51.9 tokens per second against 53.8. The draft length and probability cutoff are fixed when the server starts, so changing them means a restart.

Title: Decode speed in tokens per second, MTP head against DFlash 2
Code with MTP head: 80.5 t/s
Code with DFlash 2: 95.1 t/s
List with MTP head: 91.4 t/s
List with DFlash 2: 127.0 t/s
Geomean with MTP head: 75.1 t/s
Geomean with DFlash 2: 87.9 t/s

• License: Apache-2.0
• Cost to run: no API key or paid service is named, and the drafter files are 334 MiB and 622 MiB
• Data it reaches: the download comes from Hugging Face, and the model works on your prompts on your own machine
• Not for: standalone use, or llama.cpp builds older than version 0.8.0

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 603M
• Weights: about 302 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 3,960
• Likes: 4
• License: apache-2.0
• Based on: incoai/Qwen3.8-Flash-Next-DFlash2
• Tags: gguf, dflash2, speculative-decoding, block-diffusion, draft-model, llama.cpp, text-generation, conversational

Links

• Model: https://huggingface.co/nazeshinjite/Qwen3.8-Flash-Next-DFlash2-GGUF
• Files: https://huggingface.co/nazeshinjite/Qwen3.8-Flash-Next-DFlash2-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download nazeshinjite/Qwen3.8-Flash-Next-DFlash2-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.