DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw

DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw

A 2.9 bits-per-weight EXL3 quantization of DeepSeek-V4.1-Flash-1017 that reads text and images. It cuts the model to about 210 GB for exllamav3 runtimes, and the card measures how closely it tracks the original.

Made by Mia-AiLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a quantized copy of deepseek-ai's DeepSeek-V4.1-Flash-1017 in the EXL3 format, made by Mia-AiLab with exllamav3 and a patch set called EAQ. It averages 2.9 bits per weight, which brings the 105B-parameter model to about 210 GB in 39 shards. Vision is included, and the card says it is not a drop-in swap for the original checkpoint.

Who it suits: Advanced users with multi-GPU or large-memory rigs who want to run DeepSeek-V4.1-Flash locally at a much smaller size than the original weights.

Industry | Who uses it | How they use it | Example
Software | Developers | Serve a large local model for coding and tool calling | A local agent backend
Research | ML researchers | Study low-bit quantization and its effect on fidelity | A comparison with another 2.9 bpw build
AI infrastructure | Hardware enthusiasts | Fit a big model on a DGX Spark pair or workstation GPUs | A two-machine serving recipe
Data work | Analysts | Ask questions about text and images on private hardware | A question about a chart image

If you want a ready-to-run download with no extra files, you can skip it, since the card says you must also fetch two shards from the base model.

Where to use it:

• Runtimes that load EXL3 DeepSeek-V4.1 checkpoints, built on exllamav3
• The TensorFold DeepSeek-V4.1-Flash-1017 recipe for two DGX Spark machines, with MODEL_ID set to this repository
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Coding and agents | Developers | Output that matches the original in 98% of top tokens on agentic and code samples
Tool calling | Agent builders | Tool-call turns scored at 97.3% top-1 agreement
Image and text questions | Analysts | Answers from a model with its vision tower left native

How to set it up:

• Python, to run the Hugging Face download tool
• About 210 GB of disk for this repository
• Two extra downloads from the base model, numbered 47 and 48, each about 95 GiB and holding the Engram tables
• An EXL3-capable runtime with DeepSeek-V4.1 support

1. Install the Hugging Face hub tools with pip.
2. Download this repository with the hf command.
3. Fetch shards 47 and 48 from the base DeepSeek-V4.1-Flash-1017 model.
4. Load the checkpoint in your EXL3 runtime.

pip install -U huggingface_hub
hf download Mia-AiLab/DeepSeek-V4.1-Flash-1017-EXL3-EAQ-v1-2.9-bpw

How to use it: Load the checkpoint in a runtime that supports EXL3 DeepSeek-V4.1 models. The card says no EAQ-specific code is needed, because the format is plain EXL3.

Example: You load the model and ask it to write a short Python function that parses a log line. It answers like the original model, and on the card's test set it picked the same top token 96.6% of the time.

For newcomers and experts: This is an advanced build, so start by reading the card and confirming that your runtime supports DeepSeek-V4.1 in EXL3. Plan for the extra Engram shards before you download.

Routed experts use 3 bits, layers 18 to 22 use 2 bits, attention uses 5 bits and the head uses 6. The multi-token prediction and DSpark tensors use 4 bits. The card measures KL divergence against the original in nats, where lower is closer, and reports 0.0137 across all 100 test conversations.

Download this repository
Add Engram shards 47 and 48
Load in an EXL3 runtime
Send text or images
Answers come back

Good to know:

• License: MIT
• Cost to run: none named, but it needs large disk and memory
• Data it reaches: downloads files from Hugging Face, then runs where you host it
• Not for: use without the two base-model Engram shards
• Limits: the card says stratum samples are small, and top-1 agreement does not prove a response is correct

Title: Top-1 agreement with the original model, in percent
All: 96.6%
Agentic: 98.0%
Code: 98.2%
Tool calling: 97.3%
Multilingual: 95.5%
Chat: 94.0%

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: exllamav3
• Parameters: 105B
• Weights: about 210 GB in I16
• Formats: Safetensors
• Downloads in the last 30 days: 8,113
• Likes: 46
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Quantized versions: 1
• Tags: exllamav3, safetensors, deepseek_v41, exl3, quantized, deepseek, multimodal, image-text-to-text, conversational

Links

• Model: https://huggingface.co/Mia-AiLab/DeepSeek-V4.1-Flash-1017-EXL3-EAQ-v1-2.9-bpw
• Files: https://huggingface.co/Mia-AiLab/DeepSeek-V4.1-Flash-1017-EXL3-EAQ-v1-2.9-bpw/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Mia-AiLab/DeepSeek-V4.1-Flash-1017-EXL3-EAQ-v1-2.9-bpw

Then follow the usage in its model card. Nothing in this kit runs by itself.