Ling-3.1-flash-DSpark

Ling-3.1-flash-DSpark

A small draft model that speeds up Ling-3.1-Flash with speculative decoding in SGLang, drafting 5 or 7 tokens at a time. For teams serving Ling-3.1-Flash on NVIDIA GPUs.

Made by AQ-MedAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Ling-3.1-flash-DSpark is a 1.1B-parameter draft model, trained by its authors in the DSpark style, that makes Ling-3.1-Flash generate faster. It works through speculative decoding: the small model proposes several tokens ahead and the large model checks them. The card tests drafts of 5 and 7 tokens and credits the SpecForge framework.

Who it suits: It suits teams that already serve Ling-3.1-Flash with SGLang on NVIDIA GPUs and want faster generation.

Industry | Who uses it | How they use it | Example
AI infrastructure | Teams serving Ling-3.1-Flash | Add the draft model to their SGLang launch command | Start the server with the DSPARK algorithm and a block size of 5
Chat and API services | Providers who care about throughput | Turn on speculative decoding for busy servers | Serve several requests at once and measure tokens per second
Inference research | People studying speculative decoding | Compare 5-token and 7-token drafting against the AR and MTP baselines | Reproduce the card's runs on GSM8K or HumanEval

You can skip it if you do not serve Ling-3.1-Flash, since the draft model is built for that target.

Where to use it:

• An SGLang server on an NVIDIA GPU machine
• Setups with CUDA 12.0 or newer and PyTorch 2.0 or newer
• Windows, macOS and Linux with Python, only to download the files

Use case | Who | What they get back
Faster generation | Service operators | Higher throughput from the same target model
Drafting tests | Researchers | Mean accepted length and tokens per second
Block-size tuning | Engineers | A choice between 5 and 7 draft tokens

How to set it up: You need these first:

• An NVIDIA GPU, CUDA 12.0 or newer, and PyTorch 2.0 or newer
• The Ling-3.1-Flash target model, available at a local path
• SGLang at the version the card pins

1. Install the pinned SGLang version with pip.
2. Put the Ling-3.1-Flash model where the launch command can reach it.
3. Start the SGLang server with the DSPARK options shown below.
4. To switch to 7 draft tokens, change the block size flag.

pip install sglang==0.5.22

python3 -m sglang.launch_server \
    --model-path /models/Ling-3.1-flash \
    --host 0.0.0.0 \
    --port 30010 \
    --mem-fraction-static 0.82 \
    --trust-remote-code \
    --tp 8 \
    --ep 8 \
    --speculative-algorithm DSPARK \
    --speculative-draft-model-path AQ-MedAI/Ling-3.1-flash-DSpark \
    --speculative-dspark-block-size 5

--speculative-dspark-block-size 7

How to use it: Once the server is running, you send requests as you would to any SGLang server. The draft model works behind the scenes. To try the longer draft, change the block size to 7 and restart.

Tool | What it does
--speculative-algorithm DSPARK | Turns on DSpark speculative decoding
--speculative-draft-model-path | Points to this draft model
--speculative-dspark-block-size | Sets how many tokens the draft model proposes, 5 or 7
--tp 8 and --ep 8 | Spread the target model over 8 devices with tensor and expert parallelism
--model-path | Points to the Ling-3.1-flash target model

Example: You start the server with a block size of 5 and send it a few chat requests at once. The card measured both mean accepted length and tokens per second this way, at concurrency 1, 8, 16 and 32.

For newcomers and experts: Speculative decoding lets a small model guess ahead so the big model can check several tokens in one go. A good first try is the launch command above with a block size of 5, then 7.

For readers who know the field, the card runs greedy decoding at temperature 0, with 5 or 7 draft tokens. It tracks mean accepted length, written as tau, plus throughput, across math, code, dialogue and knowledge tests, six benchmarks in all. It says both settings beat the AR and MTP baselines in every benchmark and concurrency pairing it tested.

Launch SGLang with the target model
Load this draft model
Draft model proposes 5 or 7 tokens
Target model checks the proposals
Accepted tokens are returned faster

Good to know:

• Results: the card's result charts are images, so the exact numbers are not in the text available here.
• Scope: the claims cover the six benchmarks and four concurrency levels the card lists.
• Hardware: the sample command spreads the target model over 8 devices.
• Counts: Hugging Face lists 1.1B parameters, about 2.3 GB in BF16.
• License: MIT.
• Cost to run: none named. You need NVIDIA GPUs and the Ling-3.1-Flash model.
• Data it reaches: the requests sent to your own server. The text available here names no outside access.
• Not for: target models other than Ling-3.1-Flash.

On Hugging Face, 13 November 2026

• Parameters: 1.1B
• Weights: about 2.3 GB in BF16; about 565 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 70
• Likes: 3
• License: mit
• Tags: safetensors, qwen3

Links

• Model: https://huggingface.co/AQ-MedAI/Ling-3.1-flash-DSpark
• Files: https://huggingface.co/AQ-MedAI/Ling-3.1-flash-DSpark/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download AQ-MedAI/Ling-3.1-flash-DSpark

Then follow the usage in its model card. Nothing in this kit runs by itself.