Qwen3.8-27B-DFlash2 by incoai

Qwen3.8-27B-DFlash2 by incoai

Speeds up Qwen3.8-27B by drafting blocks of tokens that the main model then checks, with output unchanged. A 1.9B draft model for SGLang and vLLM servers, not a standalone chat model.

Made by incoai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is the DFlash 2 draft model for Qwen3.8-27B. It is not a language model you chat with. It runs inside a speculative decoding server, proposes tokens ahead of time, and lets the main model verify them, so text comes out faster. Each pass proposes a full block of tokens, retains the best few candidates for every slot, and a small selector then chooses a single coherent path.

Who it suits: It suits teams that host Qwen3.8-27B themselves and want faster generation without changing the answers.

Industry | Who uses it | How they use it | Example
AI infrastructure | Teams that serve large models | Add the drafter to an SGLang or vLLM server | Start a Qwen3.8-27B server with the DFlash algorithm turned on
Coding and math tools | Teams serving code and math answers | Cut the wait for long generations | Serve a math or coding assistant on a single H200
Inference research | Researchers of speculative decoding | Compare drafters on acceptance length and speed | Check DFlash 2 against MTP and DSpark on the card's tasks

If you do not run Qwen3.8-27B yourself, you can skip it, because it only works as a helper for that model.

Where to use it:

• SGLang, with the DFLASH speculative algorithm
• vLLM, with a speculative config that names the dflash method
• One NVIDIA H200, which is what the card's tests used
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Faster serving | Inference teams | More output tokens per second from the same target model
Unchanged output | Teams with quality checks | The same answers as normal decoding, per the card
Drafter comparison | Researchers | Acceptance length and speedup figures for several methods

How to set it up: What you need first:

• A GPU server that can run Qwen3.8-27B
• SGLang or vLLM, installed as the model card explains
• The Hugging Face command line, if you want the files locally

1. Install SGLang or vLLM. The card installs SGLang from its GitHub source and vLLM from a specific pull request build.
2. Start the server with Qwen3.8-27B as the main model and this repository as the drafter.
3. Send requests to the server as you normally would.

pip install -U huggingface_hub
hf download incoai/Qwen3.8-27B-DFlash2

How to use it: You do not call the drafter yourself. You start a server that pairs it with the main model, then use that server as usual. SGLang takes the draft model path and a draft token count of 8, and vLLM takes a speculative config.

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 8

vllm serve Qwen/Qwen3.8-27B \
  --speculative-config '{
    "method": "dflash",
    "model": "incoai/Qwen3.8-27B-DFlash2",
    "num_speculative_tokens": 7
  }'

Tool | What it does
SGLang | Runs the DFLASH algorithm with this repository as the draft model
vLLM | Uses the dflash method through its speculative config

Example: You serve Qwen3.8-27B with this drafter and send a math word problem. The answer matches what plain decoding would give, and in the card's single-request test on GSM8K the server produced 236.1 tokens per second against 68.9 without a drafter.

For newcomers and experts: Think of it as a fast assistant that guesses the next several words, while the big model checks them in one go. A good first try is to start the server and compare its speed with and without the drafter on one question.

For experts, DFlash 2 is a block-diffusion drafter. Two-tap dynamic convolutions in its backbone stop the draft quality from fading toward the end of a block. The card says output is unchanged: greedy decoding returns exactly what the main model would, and sampling keeps its distribution. The trade is load: gains shrink as concurrency rises.

The drafter predicts a block of tokens in one pass
It keeps the top candidates at each position
A selector traces one coherent path
The main model verifies the block in one step

Good to know:

Title: GSM8K output speed at concurrency 1
Autoregressive: 68.9 tok/s
MTP: 178.5 tok/s
DSpark: 185.3 tok/s
DFlash 2: 236.1 tok/s

Title: Acceptance length, higher is better
Task | MTP | DSpark | DFlash 2
GSM8K | 5.02 | 4.36 | 5.46
MATH-500 | 4.72 | 3.92 | 5.28
HumanEval | 3.91 | 3.30 | 4.39
MBPP | 3.99 | 3.51 | 4.79
MT-Bench | 3.74 | 3.01 | 4.10

• Size: the drafter has 1.9B parameters, about 3.8 GB in BF16.
• Test setup: one NVIDIA H200 running SGLang, FlashAttention 3, a block of 8 tokens and a 4096-token output limit.
• Heavy load: at concurrency 32 the DFlash 2 speedup falls to between 1.01 and 1.45 times.
• Mirror: the checkpoint is also mirrored under the z-lab name on Hugging Face.
• License: apache-2.0
• Cost to run: no API key or paid service named. You need a GPU server.
• Data it reaches: it downloads files from Hugging Face and runs inside your own server.
• Not for: standalone use, or drafting for models other than Qwen3.8-27B.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 1.9B
• Weights: about 3.8 GB in BF16; about 962 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 429,580
• Likes: 301
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 20
• Fine-tunes: 3
• Tags: transformers, safetensors, qwen3, dflash2, speculative-decoding, block-diffusion, draft-model, sglang, vllm, text-generation

Links

• Model: https://huggingface.co/incoai/Qwen3.8-27B-DFlash2
• Files: https://huggingface.co/incoai/Qwen3.8-27B-DFlash2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download incoai/Qwen3.8-27B-DFlash2

Then follow the usage in its model card. Nothing in this kit runs by itself.