--- title: "DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/037efcc1-e5e0-465b-9448-56fd5a06fdc6" updated: "2026-11-14" --- # DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs DeepSeek-V4.1-Flash-1017 as a 2.9 bits-per-weight EXL3 quantization, a 196 GiB copy for people serving large models on their own hardware. Perplexity stays close to the original model. DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-14. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by LibertAIDAI ## About ### What it is An EXL3 quantization of DeepSeek-V4.1-Flash-1017 at 2.9 bits per weight, which brings the files down to about 196 GiB. The author reports perplexity only slightly above the unquantized model. It keeps the vision tower and the drafter, but not the two Engram tables. ### Who it suits People who serve huge models on their own hardware and need a smaller copy than the original weights. It also suits anyone who wants to study how far a model can be compressed. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | AI infrastructure | Engineers serving large models | Load the EXL3 files in a compatible serving kit | Hosting a 105B-parameter model on a small cluster | | Software | Agent developers | Run tool-using coding or chat agents locally | An agentic coding loop on private hardware | | Research | Quantization researchers | Compare perplexity against the unquantized base | Checking the code and prose perplexity figures | | Self-hosting | Enthusiasts with large-memory machines | Run a large model without a hosted service | A private chat assistant at home | You can skip it if you lack hundreds of gigabytes of storage, or if you want a ready-to-run model, since you must add the Engram tables yourself. ### Where to use it - The serving kit built for DeepSeek V4.1 Flash EXL3 on two DGX Sparks - vLLM-based kits, which use their own DeepSeek encoder - Runtimes that need a template file, such as TensorFold - Windows, macOS or Linux, to download the files | Use case | Who | What they get back | | --- | --- | --- | | Serve a large model locally | Infrastructure engineers | Chat replies from a quantized DeepSeek model | | Agent loops with tools | Developers | Agent turns with adjustable reasoning effort | | Quantization comparison | Researchers | Perplexity numbers beside the unquantized model | ### How to set it up - Python, to install the Hugging Face client - About 211 GB of free disk space for 43 safetensors shards - The two Engram table shards from the base repository, numbers 47 and 48 - A serving kit or runtime that loads EXL3 models 1. Install the Hugging Face client. 2. Download the repository files to your computer. 3. Copy the Engram table shards from the base DeepSeek repository, because they are not included here. 4. Start your serving kit or runtime with these files. ```sh pip install -U huggingface_hub hf download LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw ``` ### How to use it Chat with the model through your serving kit or runtime. The repository includes a chat template file for runtimes that require one at startup. For tool-using agents, the author tuned the reasoning effort setting, so pick a level below. | Tool | What it does | | --- | --- | | chat\_template.jinja | Template file for runtimes that need one at startup | | reasoning\_effort | Sets how much the model reasons, with levels low 50, high 75 and max 100 | | Engram tables | Taken from the base repository, shards 47 and 48 | ### Example You run an agentic coding loop with tools and leave reasoning effort at its default of high. The author's benchmark of the original weights found the same quality as max at 0.55 to 0.8 times the cost, and low also matched at 0.4 to 0.6 times. ### For newcomers and experts If you are new, know that this is a huge download meant for specialist setups, not a laptop. A first step is to read the kit's own guide and confirm your hardware fits. If you know the field: it uses the EXL3 trellis format with a mul1 codebook and output channel scales. Routed experts sit at 3 bpw, layers 18 to 22 at 2 bpw, the shared expert and attention at 4 to 5 bpw, and the head at 6 bpw. The author notes that each turn's reasoning is re-read by later turns, so reasoning cost compounds. Base model weights → EXL3 quantization → This repository → Add Engram tables → Serving kit ### Good to know The author measured perplexity on held-out text. Code goes up by 1.8% and prose by 0.4% against the unquantized model. Perplexity, lower is better | Label | Value | | --- | --- | | Code, this copy | 1.9094 ppl | | Code, unquantized | 1.8749 ppl | | Prose, this copy | 6.4520 ppl | | Prose, unquantized | 6.4277 ppl | - Size: 210.7 GB in 43 safetensors shards - Included: the vision tower in BF16 and the MTP and DSpark drafter - Missing: the Engram tables, which changed in this release - License: MIT - Cost to run: none named - Data it reaches: files downloaded to your computer, nothing else named - Not for: running as is, since the Engram tables must come from the base repository ## README ````markdown # DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw Runs DeepSeek-V4.1-Flash-1017 as a 2.9 bits-per-weight EXL3 quantization, a 196 GiB copy for people serving large models on their own hardware. Perplexity stays close to the original model. Made by LibertAIDAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: An EXL3 quantization of DeepSeek-V4.1-Flash-1017 at 2.9 bits per weight, which brings the files down to about 196 GiB. The author reports perplexity only slightly above the unquantized model. It keeps the vision tower and the drafter, but not the two Engram tables. Who it suits: People who serve huge models on their own hardware and need a smaller copy than the original weights. It also suits anyone who wants to study how far a model can be compressed. ```table Industry | Who uses it | How they use it | Example AI infrastructure | Engineers serving large models | Load the EXL3 files in a compatible serving kit | Hosting a 105B-parameter model on a small cluster Software | Agent developers | Run tool-using coding or chat agents locally | An agentic coding loop on private hardware Research | Quantization researchers | Compare perplexity against the unquantized base | Checking the code and prose perplexity figures Self-hosting | Enthusiasts with large-memory machines | Run a large model without a hosted service | A private chat assistant at home ``` You can skip it if you lack hundreds of gigabytes of storage, or if you want a ready-to-run model, since you must add the Engram tables yourself. Where to use it: - The serving kit built for DeepSeek V4.1 Flash EXL3 on two DGX Sparks - vLLM-based kits, which use their own DeepSeek encoder - Runtimes that need a template file, such as TensorFold - Windows, macOS or Linux, to download the files ```table Use case | Who | What they get back Serve a large model locally | Infrastructure engineers | Chat replies from a quantized DeepSeek model Agent loops with tools | Developers | Agent turns with adjustable reasoning effort Quantization comparison | Researchers | Perplexity numbers beside the unquantized model ``` How to set it up: - Python, to install the Hugging Face client - About 211 GB of free disk space for 43 safetensors shards - The two Engram table shards from the base repository, numbers 47 and 48 - A serving kit or runtime that loads EXL3 models 1. Install the Hugging Face client. 2. Download the repository files to your computer. 3. Copy the Engram table shards from the base DeepSeek repository, because they are not included here. 4. Start your serving kit or runtime with these files. ```sh pip install -U huggingface_hub hf download LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw ``` How to use it: Chat with the model through your serving kit or runtime. The repository includes a chat template file for runtimes that require one at startup. For tool-using agents, the author tuned the reasoning effort setting, so pick a level below. ```table Tool | What it does chat_template.jinja | Template file for runtimes that need one at startup reasoning_effort | Sets how much the model reasons, with levels low 50, high 75 and max 100 Engram tables | Taken from the base repository, shards 47 and 48 ``` Example: You run an agentic coding loop with tools and leave reasoning effort at its default of high. The author's benchmark of the original weights found the same quality as max at 0.55 to 0.8 times the cost, and low also matched at 0.4 to 0.6 times. For newcomers and experts: If you are new, know that this is a huge download meant for specialist setups, not a laptop. A first step is to read the kit's own guide and confirm your hardware fits. If you know the field: it uses the EXL3 trellis format with a mul1 codebook and output channel scales. Routed experts sit at 3 bpw, layers 18 to 22 at 2 bpw, the shared expert and attention at 4 to 5 bpw, and the head at 6 bpw. The author notes that each turn's reasoning is re-read by later turns, so reasoning cost compounds. ```flow Base model weights EXL3 quantization This repository Add Engram tables Serving kit ``` Good to know: The author measured perplexity on held-out text. Code goes up by 1.8% and prose by 0.4% against the unquantized model. ```chart Title: Perplexity, lower is better Code, this copy: 1.9094 ppl Code, unquantized: 1.8749 ppl Prose, this copy: 6.4520 ppl Prose, unquantized: 6.4277 ppl ``` - Size: 210.7 GB in 43 safetensors shards - Included: the vision tower in BF16 and the MTP and DSpark drafter - Missing: the Engram tables, which changed in this release - License: MIT - Cost to run: none named - Data it reaches: files downloaded to your computer, nothing else named - Not for: running as is, since the Engram tables must come from the base repository ## On Hugging Face, 14 November 2026 - Task: text generation - Parameters: 105B - Weights: about 211 GB in I16 - Formats: Safetensors - Downloads in the last 30 days: 7,382 - Likes: 60 - License: mit - Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017 - Quantized versions: 3 - Fine-tunes: 3 - Tags: safetensors, deepseek_v41, exl3, exllamav3, deepseek, quantized, text-generation, conversational ## Links - Model: https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw - Files: https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw on Hugging Face](https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | DeepSeek-V4.1-Flash-1017 EXL3 2.9 bpw (this page) | 7,382 | 60 | 105B | mit | 2026-10-19 | | [DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw](https://mindket.com/p/e2aa9049-7d40-46f6-b187-57f9ffaaccf1) | 8,113 | 46 | 105B | mit | 2026-10-20 | | [DeepSeek-V4.1-Flash-1017-EXL3-franken-v100](https://mindket.com/p/91bd8bd6-2fba-4896-ae7f-5c8ff314c941) | 313 | 7 | — | mit | 2026-10-26 | | [DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000](https://mindket.com/p/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e) | 2,514 | 42 | 219B | mit | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017 TrellisMX K2 Mix](https://mindket.com/p/07ee61ca-5fd5-44fb-a019-f885468d68a4) | 338 | 12 | 115B | mit | 2026-10-20 | | [GLM-5.3-Flash EXL3 3.05bpw](https://mindket.com/p/783914ac-53d0-4c6b-adb4-69db026aa354) | 0 | 2 | — | mit | 2026-10-12 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-14. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-19 - First published: 2026-10-17 - Checked on Hugging Face: 2026-11-14 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw on Hugging Face](https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/037efcc1-e5e0-465b-9448-56fd5a06fdc6) - [Its page on Mindket](https://mindket.com/p/037efcc1-e5e0-465b-9448-56fd5a06fdc6)