DeepSeek-V4.1-Flash-1017 TrellisMX K2 Mix

DeepSeek-V4.1-Flash-1017 TrellisMX K2 Mix

A compressed copy of DeepSeek-V4.1-Flash-1017 sized for two 96 GB GPUs, with a few experts stored at higher precision. A research build for people who serve large models.

Made by brandonmusic on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a compressed copy of DeepSeek-V4.1-Flash-1017, a 115B text generation model, built by brandonmusic with a method called TrellisMX. Most routed experts are stored at 2 bits and a few at 3 or 4 bits, which suits two RTX PRO 6000 Blackwell cards with 96 GB each. The card rates quality by KL divergence from the official checkpoint, where lower means closer.

Who it suits: Engineers and researchers who run large language models on their own multi-GPU machines and want to know how closely a compressed copy tracks the original.

Industry | Who uses it | How they use it | Example
AI infrastructure | Inference engineers | Test low-bit expert storage on two-GPU servers | Check whether the layout loads on their own serving image
Research | Quantization researchers | Compare per-expert bit widths with the official checkpoint | Read the measured KL figures for this build against the K2 build
Hardware labs | GPU lab teams | Try a large mixture-of-experts model on workstation cards | Test the author's serving image on two 96 GB cards

Anyone who needs a model that is ready to serve today can skip it, since whole-model serving, speed and cache capacity are not measured yet.

Where to use it: It targets multi-GPU Blackwell machines, and the files download on any common desktop system.

• Two-GPU Blackwell workstations running tensor parallel 2
• The author's test serving image, which reads this layout
• Two DGX Spark (GB10) nodes, through an unvalidated test kit in the sm121-tp2 folder
• Windows, macOS or Linux, for downloading the files with Python tools

Use case | Who | What they get back
Fidelity checks | Researchers | Measured KL divergence against the official checkpoint
Layout testing | Inference engineers | A checkpoint pre-sliced for tensor parallel 2
Compression study | Quantization researchers | Per-layer expert widths listed in a manifest file

How to set it up: Fetching the files is easy, and serving them needs a matching setup.

• Python with pip, to fetch the files
• About 115 GB of disk space for the weights
• Two 96 GB Blackwell cards if you plan to serve it
• A serving setup that reads this layout, such as the author's test image

1. Install the Hugging Face hub tools with pip.
2. Download the checkpoint with the hf command.
3. Point a serving image that reads this TrellisMX layout at the downloaded folder.

pip install -U huggingface_hub
hf download brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix

How to use it: The card gives no serve command for this checkpoint, so day-to-day use today means reading the stored layout and the measured results.

Tool | What it does
trellismx/manifest.json | Lists the bit width of every expert in each layer
tp2 sidecar files | Hold the tensors for each width in a layer, pre-sliced for two GPUs
sm121-tp2 folder | An unvalidated test kit for two DGX Spark nodes
Test serving image | Reads this layout and has passed layer-level GPU tests

Example: You ask how much closer this mix sits to the official model than the plain K2 build. The card answers with numbers. On 2,048-token sequences the mean KLD is 0.1306 with a top-1 match of 87.56%, against 0.1429 for K2.

For newcomers and experts: This is a huge language model squeezed so it fits on two high-end graphics cards. It is a research release, so the first small thing to try is downloading the files and reading the measured numbers. Chatting with it needs a server setup that the card does not describe yet.

TrellisMX stores each routed expert as trellis codes and decodes them to FP8 (E4M3) operands, with UE8M0 scales per 32 values in a lossless CSF format. Each width came from an allocation that favors the biggest KL gain per byte, inside a size budget that lets the two cards still hold a KV cache of roughly 3 million tokens. The wider experts reuse the K3 and K4 encodes, re-laid out for tensor parallel 2.

Start from the official checkpoint
Encode experts at 2, 3 or 4 bits
Choose each width by KL per byte
Slice the layout for two GPUs
Run layer tests and measure KLD

Good to know: Only 629 of the 15,360 routed experts are stored wider than 2 bits, and the card compares three builds.

Title: Mean KLD on 2,048-token sequences, lower is closer
K2 Mix: 0.1306 nats
K2: 0.1429 nats
K3: 0.0664 nats

• Status: a research build, and only KLD has been measured so far.
• Size: the routed experts add 2.87 GB over the K2 build. The K3 build needs three cards.
• License: MIT, the same as the base model.
• Cost to run: none named, though you need two 96 GB GPUs.
• Data it reaches: the files you download to your disk. The card names no web or account access.
• Not for: ready-to-serve use today. The whole model has not booted, so serving, speed and cache capacity are unmeasured, and the published K2 image does not serve this mix.

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 115B
• Weights: about 115 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 338
• Likes: 12
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Tags: safetensors, deepseek_v41, trellismx, trellis-quantization, mixed-precision, 2-bit, 3-bit, 4-bit, deepseek, deepseek-v4.1, engram, text-generation

Links

• Model: https://huggingface.co/brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix
• Files: https://huggingface.co/brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download brandonmusic/DeepSeek-V4.1-Flash-1017-TrellisMX-K2-Mix

Then follow the usage in its model card. Nothing in this kit runs by itself.