Kimi-K3.1-Dflash2-Preview

Kimi-K3.1-Dflash2-Preview

Speeds up Kimi-K3.1-Pro serving by drafting blocks of tokens that the main model checks in one pass. A draft model for SGLang users, used alongside the target and not a standalone language model.

Made by RadixArk on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds a DFlash2 draft model that goes with Kimi-K3.1-Pro. Paired with SGLang, it lets the large model generate faster by speculative decoding. The draft proposes a block of tokens at once, and the main model verifies the block in a single forward pass. The repository holds only the BF16 draft weights and configuration.

Who it suits: Teams who already serve Kimi-K3.1-Pro with SGLang and want faster generation.

Industry | Who uses it | How they use it | Example
AI platforms | Inference engineers | Add a draft model to speed up serving | Raise tokens per second on a Kimi deployment
Research | Systems researchers | Study block-wise speculative decoding | Compare acceptance length at widths 5 and 8
Software | Backend developers | Tune a self-hosted model server | Test the draft on math and science prompts

If you do not run Kimi-K3.1-Pro yourself, you can skip it, since the draft does nothing alone.

Where to use it:

• SGLang, the serving framework the card targets
• A multi-GPU server, as the card's example uses 8 GPUs

Use case | Who | What they get back
Faster serving | Inference engineers | More tokens accepted per verification step
Benchmarking | Researchers | Acceptance length measurements
Self-hosting Kimi | Platform teams | A drop-in draft for the target model

How to set it up:

• Python with pip, to fetch the files
• The target model, Kimi-K3.1-Pro, which supplies the embeddings, output head, tokenizer and chat template
• SGLang and a multi-GPU machine, since the card's measurements ran on one 8-GPU node

1. Install the Hugging Face hub tool.
2. Download the draft files.
3. Launch SGLang with the target model and this draft, following the card's example.

pip install -U huggingface_hub
hf download RadixArk/Kimi-K3.1-Dflash2-Preview

How to use it: You do not call this model yourself. You start an SGLang server for Kimi-K3.1-Pro with the draft attached, then send requests to that server as usual. The part of the card shown here cuts off before the full launch command, so copy it from the model card.

Example: You send a math question to your SGLang server for Kimi-K3.1-Pro. The draft proposes several tokens at once and the target accepts the ones it agrees with, so the answer arrives in fewer steps.

For newcomers and experts: If you are new, think of the draft as a fast assistant guessing the next few words while the big model checks them. It only matters if you run the large model on your own servers.

For experts, the draft has five MLA layers fed by hidden states from six of the target's layers. Its attention is a block-causal sliding window of 1,024 tokens. A low-rank selector then chooses each proposed token from the top 32 candidates.

The draft reads the target's hidden states
It proposes a block of tokens in parallel
The target verifies the block in one pass
Accepted tokens plus one target token commit

Good to know: The card measures acceptance length, the number of tokens committed per verification step. It does not measure answer accuracy.

Title: Acceptance length from the card
Dataset | Width 5 | Width 8
GSM8K | 4.259405 | 5.780277
AIME26 | 3.231564 | 3.696036
GPQA Diamond | 3.275916 | 3.768280

• License: Apache 2.0, as the Hugging Face record lists it.
• Cost to run: no API key or account is named. You need the target model and a multi-GPU server.
• Data it reaches: downloading reaches Hugging Face. The draft takes its embeddings and tokenizer from the target model.
• Not for: standalone use. The card says it is not a language model on its own, and it does not establish quality across the full 1M context.

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 2.8B
• Weights: about 5.6 GB in BF16; about 1.4 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,682
• Likes: 6
• License: apache-2.0
• Based on: moonshotai/Kimi-K3.1-Pro
• Tags: safetensors, qwen3, speculative-decoding, dflash2, sglang, kimi, text-generation

Links

• Model: https://huggingface.co/RadixArk/Kimi-K3.1-Dflash2-Preview
• Files: https://huggingface.co/RadixArk/Kimi-K3.1-Dflash2-Preview/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download RadixArk/Kimi-K3.1-Dflash2-Preview

Then follow the usage in its model card. Nothing in this kit runs by itself.