Ling-3.1-flash

Ling-3.1-flash

Handles complex agent, search, office and coding work with 560B total and 25B active parameters and a 1M-token context. Serving examples use SGLang on multi-GPU servers, and the weights are open under MIT.

Made by inclusionAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Ling-3.1-flash is an upgraded hybrid reasoning model from inclusionAI for demanding practical work. It has 560B parameters in total, uses 25B of them for each token, and reads a context of 1M tokens. It improves on Ling-3.0-flash in its architecture and in tasks such as general agents, search, office productivity and software engineering.

Who it suits: It suits teams with multi-GPU servers who want an open model for coding agents, office automation and tool use.

Industry | Who uses it | How they use it | Example
Software engineering | Teams that run coding agents | Serve the model behind an agent tool | Point Claude Code or Kilo Code at the served model
Office productivity | Teams that automate office work | Let the model handle multi-step tasks with tools | Run an office task that needs several tool calls
Healthcare and finance research | Researchers in these fields | Test the model on domain questions | Try questions from the card's healthcare and finance benchmarks

If you do not have a multi-GPU server, you can skip it, since even the 4-bit weights are about 286 GB.

Where to use it:

• SGLang, which the card recommends, through a Docker image with support for this model
• vLLM, which the card also covers
• Agent tools such as Claude Code and Kilo Code, where the card says the model works well, as it does in Hermes Agent and OpenClaw
• The ModelScope and OpenRouter pages that the card links to
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Coding agents | Developers | Help with long coding tasks and terminal tool use
Search and research agents | Agent builders | Multi-step answers that call tools along the way
Domain questions | Analysts | Answers in STEM, healthcare and finance

How to set it up: What you need first:

• Docker with GPU access
• A multi-GPU server, such as 8 x H200 for the card's example
• Enough disk for about 572 GB of FP8 weights

1. Pull the SGLang image that supports this model.
2. Start the server with the model path and the parallel settings for your GPUs.
3. Send chat requests to the server on port 30000.

docker pull lmsysorg/sglang:dev-ling-3.1-flash

docker run --gpus all --network host --ipc=host --rm \
  lmsysorg/sglang:dev-ling-3.1-flash \
  python3 -m sglang.launch_server \
    --model-path inclusionAI/Ling-3.1-flash \
    --tp 8 --ep 8 \
    --speculative-algorithm NEXTN \
    --host 0.0.0.0 --port 30000

To fetch the files yourself, use the Hugging Face tools.

pip install -U huggingface_hub
hf download inclusionAI/Ling-3.1-flash

How to use it: After the server starts, send it a chat completions request with the model name Ling-3.1-flash. The card's example turns thinking on, streams the reply, and sets temperature 1.0, top_k 20 and top_p 0.95. On 4 x GB300, use the smaller parallel settings from the card instead.

The optional Hopper decode optimizations make batch-1 decode about 10% faster. You enable them with these two settings.

SGLANG_ENABLE_FP8_BLOCK_GEMV=1
SGLANG_ENABLE_FUSED_KDA_OUTPUT_QUANT=1

Tool | What it does
SGLang | Serves the model with NEXTN speculative decoding and Hopper decode optimizations
vLLM | The card's second serving option, with its own setup steps
LingCode | A coding framework built on the open-source agent Pi, which the authors may release later

Example: You start the server on 8 H200 cards and ask a short factual question with thinking on and streaming enabled. The server streams the answer back as chat completion chunks.

For newcomers and experts: This model needs a GPU cluster, so a first try is easier through the hosted pages the card links to. If you run it yourself, start with the SGLang image and one short question.

For experts, the design is a hybrid-linear mixture of experts with 63 KDA layers and 9 gated MLA layers in a 7 to 1 ratio, plus 4 dense layers. It has 512 routed experts, 1 shared expert and 8 activated experts per token, with 64 attention heads and a hidden size of 4096. The trade is memory: only 25B parameters work per token, but all 560B must be loaded.

Pull the SGLang image
Start the server with parallel settings
Send a chat request on port 30000
Stream the reply back

Good to know:

Title: Parameters in billions
Total: 560 billion
Active per token: 25 billion

• Hardware: the example server uses 8 x H200 with TP8 and EP8, or 4 x GB300 with the smaller settings.
• Size: the files weigh about 572 GB in FP8 form, and about 286 GB at 4-bit, which is an estimate.
• Benchmarks: the card names tests such as Terminal-Bench 4.0, SWE-bench Pro and MCP-Atlas, and shows the scores only in images.
• LingCode: it is not released, and the authors say only that they are considering it.
• License: mit
• Cost to run: no API key or paid service named. The GPU hardware is the cost.
• Data it reaches: it pulls a Docker image and the model files, and the example server listens on every network interface on port 30000.
• Not for: small machines, since the example needs 8 x H200 or 4 x GB300.

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 572B
• Weights: about 572 GB in F8_E4M3; about 286 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 8,327
• Likes: 170
• License: mit
• Quantized versions: 8
• Tags: safetensors, bailing_hybrid, text-generation, conversational, fp8

Links

• Model: https://huggingface.co/inclusionAI/Ling-3.1-flash
• Files: https://huggingface.co/inclusionAI/Ling-3.1-flash/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download inclusionAI/Ling-3.1-flash

Then follow the usage in its model card. Nothing in this kit runs by itself.