Qwen3.8-Flash-Next-DFlash2

Qwen3.8-Flash-Next-DFlash2

Speeds up Qwen3.8-Flash-Next by drafting several tokens at once for the main model to check, for engineers who serve that model with SGLang and want faster output without changing it.

Made by incoai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds DFlash 2, a small draft model for speculative decoding with Qwen/Qwen3.8-Flash-Next. It cannot answer prompts on its own. Inside a serving setup it guesses tokens ahead, and the main model checks them. Output stays identical to the main model's, so you get speed without a change in answers.

Who it suits: It suits engineers who host Qwen3.8-Flash-Next themselves and want higher throughput from the same hardware.

Title: Who uses this model
Industry | Who uses it | How they use it | Example
AI infrastructure | Inference engineers | Add a draft model to an SGLang server | Start the server with both models and compare speed
Software | Teams hosting their own model | Cut the time a chat reply takes | Serve a coding assistant faster for several users
Research | Machine learning researchers | Study block-diffusion drafting | Compare acceptance length against the built-in MTP drafter

If you do not serve Qwen3.8-Flash-Next yourself, you can skip it, since it does nothing without that model.

Where to use it:

• SGLang, as the speculative decoding server
• A machine with an NVIDIA GPU, since the card tested one GB300
• Windows, macOS or Linux with Python, for downloading the files

Title: Jobs it does
Use case | Who | What they get back
Faster replies | Teams serving the model | More output tokens per second than plain decoding
Higher throughput | Operators | Better speed with 1, 8 or 32 requests at once
Research runs | Researchers | Acceptance lengths to compare drafters

How to set it up: You need a working SGLang install and the main model.

• Python with pip, git and the hf command
• An NVIDIA GPU
• The target model, Qwen/Qwen3.8-Flash-Next-FP8

1. Clone the SGLang repository from its project page.
2. Move into the folder and check out the commit the card names.
3. Apply the small capture patch stored in this repository, which Flash-Next needs until support lands upstream.
4. Install SGLang with pip.
5. Launch the server with the target model and this draft model.

cd sglang
git checkout 3c560089e1818a5217a73f2168273485a5350341
git apply "$(hf download incoai/Qwen3.8-Flash-Next-DFlash2 \
  patches/qwen38-dflash2-capture.patch --quiet)"
pip install -e "python[all]"

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-Flash-Next-FP8 \
  --linear-attn-verify-backend triton \
  --mamba-ssm-dtype bfloat16 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-Flash-Next-DFlash2 \
  --speculative-draft-model-quantization unquant \
  --speculative-num-draft-tokens 4

How to use it: Once the server runs, you send requests to it as you would to any SGLang server. The draft model works behind the scenes and you do not call it directly.

Title: Main commands
Tool | What it does
hf download | Fetches the capture patch from this repository
git apply | Applies that patch to your SGLang checkout
sglang.launch_server | Starts the server with the main model and this drafter

Example: You start the server with Qwen3.8-Flash-Next-FP8 as the target and DFlash 2 as the drafter, then send a math question. The reply streams back faster than plain decoding, and the answer matches what the target alone would give.

For newcomers and experts: Speculative decoding works like a fast typist and a careful editor. The small model guesses a few words, and the big model approves or corrects them. This repository is the fast typist, so you start by launching the server command above.

DFlash 2 is a block-diffusion drafter. It predicts a block of tokens in one pass and keeps the top candidates at each position, then a light selector traces one coherent path through them. Dynamic convolutions with two taps in the backbone stop the draft quality from fading at the block's tail. Decoding is lossless: greedy output equals the target's, and sampling keeps its distribution. The block size is 4, limited by the current QSA implementation.

Drafter predicts a block of tokens
Selector picks one coherent path
Target model verifies the draft
Accepted tokens become the output

Good to know: The card tested on one NVIDIA GB300 with an FP8 target. At concurrency 1 on GSM8K, DFlash 2 averaged 3.26 tokens accepted per verification step, against 3.08 for MTP. Other people have published 5 quantized versions and 1 fine-tune.

Title: GSM8K output speed at concurrency 1
Autoregressive: 205.4 tok/s
MTP: 326.0 tok/s
DFlash 2: 368.2 tok/s

• License: Apache 2.0.
• Cost to run: none named, apart from an NVIDIA GPU.
• Data it reaches: it downloads files from Hugging Face, and the SGLang source comes from its own project page.
• Not for: standalone use, since it only drafts tokens for Qwen/Qwen3.8-Flash-Next.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 603M
• Weights: about 1.2 GB in BF16; about 302 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 58,226
• Likes: 84
• License: apache-2.0
• Based on: Qwen/Qwen3.8-Flash-Next
• Quantized versions: 5
• Fine-tunes: 1
• Tags: transformers, safetensors, qwen3, dflash2, speculative-decoding, block-diffusion, draft-model, sglang, text-generation

Links

• Model: https://huggingface.co/incoai/Qwen3.8-Flash-Next-DFlash2
• Files: https://huggingface.co/incoai/Qwen3.8-Flash-Next-DFlash2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download incoai/Qwen3.8-Flash-Next-DFlash2

Then follow the usage in its model card. Nothing in this kit runs by itself.