Flint-DFlash2

Flint-DFlash2

A 403M-parameter draft model that speeds up serving of the Flint language model with speculative decoding in SGLang, without changing its output. It is not a standalone model.

Made by incoai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Flint-DFlash2 is the DFlash 2 draft model for the Flint language model. It is not a language model you chat with on its own. It only works in a serving setup built on speculative decoding, where it guesses upcoming tokens and the main model verifies them.

DFlash 2 is a block-diffusion drafter. In one forward pass it guesses an entire run of upcoming tokens and offers several candidates per position. A small selector then chooses one consistent path.

Who it suits: Engineers who serve the Flint model and want more tokens per second on the same GPU. The card reports that the output stays identical to normal decoding.

Industry | Who uses it | How they use it | Example
AI infrastructure | Inference engineers | Add a draft model to a serving stack | Serving Flint with drafting turned on
Software | Teams hosting their own LLM | Raise throughput without changing answers | A coding assistant that replies faster
Research | Decoding researchers | Study block-diffusion drafting | Comparing acceptance length across tasks

You can skip it if you do not run the Flint model, or if you want a model to chat with directly.

Where to use it: It works inside an SGLang server that runs the Flint model.

• SGLang, installed from the commit the card validated
• A CUDA-class GPU, since the card measured on one NVIDIA GB300
• Linux or Windows with Python to fetch the files

Use case | Who | What they get back
Faster serving | Inference engineers | More output tokens per second than normal decoding
Same answers | Application teams | Greedy output identical to the target model
Concurrent load | Platform teams | Gains at 1, 8 and 32 simultaneous requests

How to set it up: You need Python, a GPU server, and a Flint-capable SGLang build. The card says Flint support sits in an open SGLang pull request, so you install SGLang at the commit the authors validated.

• Python with pip
• A GPU suited to serving the Flint model
• The RecursiveSI/flint target model

1. Clone the SGLang repository from GitHub and enter the folder.
2. Fetch and check out the validated commit, then install SGLang in editable mode.
3. Start the server with the Flint model and this draft model.

cd sglang
git fetch origin 37bc771d2b253e2d87d27ace71297154874ee3b4
git checkout 37bc771d2b253e2d87d27ace71297154874ee3b4
pip install -e "python[all]"

sglang serve RecursiveSI/flint \
  --enable-fp32-lm-head \
  --reasoning-parser flint \
  --tool-call-parser flint \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Flint-DFlash2

How to use it: Once the server is up, send requests to it as you would to any SGLang server. Drafting happens behind the scenes, and the target model verifies every drafted token.

Tool | What it does
sglang serve | Starts the model server
--enable-fp32-lm-head | Uses a 32-bit float output head, as the flag name says
--reasoning-parser flint | Reads Flint's reasoning output format
--tool-call-parser flint | Reads Flint's tool-call format
--speculative-algorithm DFLASH | Switches on DFlash speculative decoding
--speculative-draft-model-path | Points the server at this draft model

Example: You start the server with the command above and send a math word problem. On GSM8K at one request at a time, the card measured 686.5 output tokens per second against 316.7 for normal decoding.

For newcomers and experts: If you are new, think of it as a helper that guesses the next several words so the main model can check them in one go. It only helps when you run Flint on your own server.

If you know the field, the drafter proposes a block of 8 positions, with 7 draft tokens per verification step. Two-tap dynamic convolutions keep quality from decaying late in the block. Decoding is lossless, since greedy output matches the target and sampling keeps its distribution.

Drafter predicts a block of tokens
Selector picks one coherent path
Flint verifies the drafted tokens
Accepted tokens are kept
Next block starts from there

Good to know: The card measured acceptance lengths from 3.49 on MT-Bench to 4.53 on HumanEval. Speedups were highest at one request at a time and stayed above 1.4 times at 8 and 32 requests.

Title: Speedup over normal decoding at one request, times
HumanEval: 2.32 times
MBPP: 2.25 times
GSM8K: 2.17 times
MATH-500: 2.06 times
MT-Bench: 1.77 times

• License: Apache 2.0
• Cost to run: none named, but the card measured on an NVIDIA GB300 GPU
• Data it reaches: the model files you download and the requests you send your own server
• Not for: standalone use, since it only drafts tokens for the Flint target model

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 403M
• Weights: about 805 MB in BF16; about 201 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,651
• Likes: 9
• License: apache-2.0
• Based on: RecursiveSI/flint
• Quantized versions: 1
• Tags: transformers, safetensors, qwen3, dflash2, speculative-decoding, block-diffusion, draft-model, sglang, text-generation

Links

• Model: https://huggingface.co/incoai/Flint-DFlash2
• Files: https://huggingface.co/incoai/Flint-DFlash2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download incoai/Flint-DFlash2

Then follow the usage in its model card. Nothing in this kit runs by itself.