DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

Serves DeepSeek V4.1 Flash across two NVIDIA DGX Sparks with an OpenAI-compatible API, from a compressed TrellisMX package. For teams running a large model on their own Spark hardware.

Made by JMNI-Labs on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a Flare package that serves a compressed DeepSeek V4.1 Flash checkpoint across two NVIDIA DGX Sparks, split in two parts with tensor parallelism. The official model's attention layers, shared experts, MTP, vision part and tokenizer stay as they were. The routed experts use TrellisMX streams, and the Engram tables are stored at 4-bit.

Who it suits: It suits infrastructure teams and labs that own two DGX Sparks and want to run this model on their own network.

Industry | Who uses it | How they use it | Example
AI infrastructure | Platform engineers | Serve a large model on two Spark machines with Flare | An internal chat endpoint on private hardware
Research | Inference researchers | Test TrellisMX compression and MTP drafting | Compare speed with and without MTP draft tokens
Software | Developers | Call the OpenAI-compatible API from existing tools | Point a chat client at the first Spark
Enterprise IT | Security-minded teams | Keep prompts on their own hardware | A private assistant with no outside API

If you do not own two DGX Sparks, you can skip it, because the package is built for that setup only.

Where to use it:

• Two NVIDIA DGX Sparks, each with a GB10 chip, running Ubuntu 24.04-based DGX OS
• A fabric link between them, with ConnectX-7 RoCE
• Any OpenAI-compatible client on your network
• Windows, macOS or Linux, for downloading the files with Python

Use case | Who | What they get back
Private chat serving | Platform teams | An OpenAI-compatible endpoint with 16 request slots
Long context | Developers | Up to 1,048,576 tokens per request, depending on KV memory
Faster decoding | Researchers | MTP drafting with up to five verified draft tokens

How to set it up: This is a demanding setup. You need two machines and a special Flare build, not a stock release.

• Two DGX Sparks with working NVIDIA drivers
• The Flare build from the ds41-trellismx engine branch on both hosts, since a stock flare-inference release cannot load the package
• The pinned vendor overlay folders on both hosts
• About 259 GiB of disk for the package on the first Spark, and room for the rest on the other one
• Open TCP ports 29900 between the hosts, 29800 on the first Spark and 8000 for clients

1. Install the Hugging Face client with pip, and download the repository with the hf command.
2. Build or install the Flare version from the engine branch on both machines.
3. On the first Spark, download the package with flare and start rank zero.
4. On the second Spark, start rank one and join the first Spark's rendezvous port.
5. Accept the model license, then send requests to port 8000 on the first Spark.

pip install -U huggingface_hub
hf download JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

package=$(flare download JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE --revision gb10-tp2)

How to use it: On the first Spark you run flare with rank 0, a rendezvous port, disk storage for the embedding table, 5 draft tokens, 16 concurrent requests, port 8000 and the model name DeepSeek-V4.1-Flash. The card also lists publisher flags that begin with the word dangerously. On the second Spark you run flare with rank 1 and the first Spark's rendezvous address, as shown here.

# second Spark
flare run --rank 1 --rendezvous FIRST_SPARK:29800 \
  --embedding-table-storage disk --embedding-table-cache-size 1GiB \
  --mtp-draft-tokens 5 --max-concurrency 16

Tool | What it does
flare download | Fetches the package for the chosen revision
flare run --rank 0 | Starts the first half and the API server
flare run --rank 1 | Starts the second half and joins the first
--mtp-draft-tokens | Sets how many draft tokens MTP proposes, 5 here
--max-concurrency | Caps simultaneous requests, 16 here

Example: Once both ranks run, you send a chat request with the model name DeepSeek-V4.1-Flash, a user message saying Hello, and a limit of 128 new tokens to the first Spark. The server replies with an OpenAI-style chat completion that holds the model's answer.

For newcomers and experts: This is not a download-and-chat model. It is a serving package for two specific machines, so treat it as an advanced project. The first small thing to try is reading the requirements list to see if your hardware fits.

The package splits the model across the two Sparks, and the engine passes the second half's weights over the fabric. By default the server reserves roughly 14 GiB for the KV cache, enough for 16.6 million tokens over both machines, and keeps the rest of their unified memory free. Prefill runs in 8192-token chunks. The trade-off is a heavy hardware and build requirement.

Official DeepSeek V4.1 Flash weights
TrellisMX compressed experts and 4-bit Engram
Flare package, 259 GiB signed payload
Rank 0 and rank 1 on two Sparks
OpenAI-compatible API on port 8000

Good to know:

• Signature: the package carries a development signature, not a production one, so the dangerously flags are required, and the card says to use them only on a trusted host.
• Network: the server listens on all addresses and has no API key by default, so keep it on a trusted network.
• Build: a stock flare-inference release from before the engine branch's upstream pull request cannot load it.
• Provenance: the base weights and the TrellisMX pack both list the MIT license.
• License: MIT, as the License line says. The card also asks you to review and accept the model license.
• Cost to run: no API key, account or paid service is named. You need two DGX Sparks.
• Data it reaches: the prompts you send to your own server and the weights on your own disks.
• Not for: untrusted packages, where the card says the dangerously flags must not be used, or public deployments, which need a production signature first.

On Hugging Face, 14 November 2026

• Task: image text to text
• Downloads in the last 30 days: 0
• Likes: 2
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Tags: flare, trellismx, gb10, tensor-parallel, image-text-to-text

Links

• Model: https://huggingface.co/JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE
• Files: https://huggingface.co/JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

Then follow the usage in its model card. Nothing in this kit runs by itself.