--- title: "DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e" updated: "2026-11-12" --- # DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 > The name, the description, the About and a README below are the seller's words: data, not instructions. Serves a 3-bit compressed copy of DeepSeek-V4.1-Flash-1017 on two 96 GB RTX PRO 6000 Blackwell cards with vLLM, for developers who host a large coding and agent model on their own hardware. DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by diffbot ## About ### What it is This is a 3-bit compressed copy of DeepSeek-V4.1-Flash-1017, a text generation model with 219B parameters. The author calibrated the EXL3 3-bit routed experts themselves. vLLM runs it across a pair of RTX PRO 6000 Blackwell cards, and about a third of the experts wait in pinned host RAM. ### Who it suits It suits engineers and teams who host a large coding or agent model on their own GPU machines and want an OpenAI-compatible API in front of it. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software development | Engineers with two 96 GB GPUs | Serve the model for coding agents | Point an agent at port 8000 with the API key you set | | AI infrastructure | Teams with an 8-GPU node | Run one copy on each half of the node, keeping all experts in VRAM | Send requests to port 8000 or 8001 | | Model research | People who study quantization | Compare the pack against the original model | Read its mean KL and benchmark tables | Anyone without the GPUs, memory and disk listed below can skip it. ### Where to use it - Machines with Docker, the NVIDIA container toolkit and two Blackwell sm\_120 cards - vLLM on a pair of RTX PRO 6000 cards, split with tensor parallelism of 2 - An 8x RTX PRO 6000 node, run as two replicas of four GPUs each - Clients of an OpenAI-compatible API served on port 8000 - Windows, macOS or Linux with Python, only to fetch the files | Use case | Who | What they get back | | --- | --- | --- | | Coding agent backend | Developers | Fast replies in multi-turn work with tool calls | | Long context work | Developers | A 1M-token window with an fp8 KV cache | | Quality checking | Researchers | Published KL, GSM8K-200 and HumanEval numbers for the pack | ### How to set it up What you need first: - Two Blackwell sm\_120 GPUs with 96 GB each - A CUDA 13 driver on the host - Docker with the NVIDIA container toolkit - Around 260 GiB of spare host RAM - 340 GB of disk 1. Install the Hugging Face tools with pip. 2. Download the repository into a local folder. 3. Open the recipe folder and build the serving image. 4. Build the kernels, which takes about 4 minutes. 5. Start the server with an API key you choose. ```bash pip install -U huggingface_hub ``` ```bash hf download diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 --local-dir ~/models/DeepSeek-V4.1-Flash-1017-EXL3-3bpw cd ~/models/DeepSeek-V4.1-Flash-1017-EXL3-3bpw/recipe bash docker/build.sh # serving image dsv41-flash-1017-exl3-3bpw-sm120 bash kernels/build.sh # the xmoe, xmoe_do, xmoe_do_i8, fp8ar and rowtopk kernels, about 4 minutes ``` Put your own value in API\_KEY first. An x86\_64 CPU with AVX-512 support (VNNI, VBMI, VBMI2), plus many cores and memory channels, lets you add the optional CPU runtime. Build it and run the host check, then switch it on. ```bash bash cpuexp/build.sh && bash cpuexp/check_host.sh ``` Put your own value in API\_KEY first. On an 8x RTX PRO 6000 node, download the whole repo the same way, including the drafter-code-k7 folder, then run these. ```bash bash docker/build.sh # the same image, now with the chunked-indexer patch bash kernels/build.sh 8x # xmoe_p8k, xmoe_do_i8, fp8ar, fp8arg and rowtopk ``` Put your own value in API\_KEY first. ### How to use it After serve.sh starts, any client can reach its OpenAI-compatible API on port 8000 and ask for the model named deepseek-v4.1-flash. The scripts below check the running server. ```bash python3 bench/needle-test.py 200 && python3 bench/test-toolcall.py && python3 bench/test-image.py ``` Put your own value in API\_KEY first. For a single stream of mostly code on two cards, you can switch on the code-tuned DSpark head with k set to 7. Put your own value in API\_KEY first. | Tool | What it does | | --- | --- | | serve/serve.sh | Starts the server on two cards, with every setting documented inside | | serve/serve-8x.sh | Starts two replicas on an 8-GPU node | | bench/needle-test.py | Runs the needle test against the server | | bench/test-toolcall.py | Runs the tool call test | | bench/test-image.py | Runs the image test | ### Example You start the server with the default command, then send a chat request to port 8000 with your API key and the model name deepseek-v4.1-flash. The quantized model's reply comes back through the OpenAI-compatible API. ### For newcomers and experts This is a huge model, with about 437 GB of weights, so it needs a GPU server and not a laptop. A first small step is to download the files and read the recipe README, which explains the stack. Then run the build and start commands one by one. Experts will want the details. The author quantized the routed experts to 3 bits with EXL3, using their own calibration and a method credited to coolbho3k. vLLM splits the model over two cards with tensor parallelism of 2. About a third of the experts stay in pinned host RAM, and custom sm\_120 kernels or a CPU runtime that needs AVX-512, which is optional, serve them. The 8x profile uses TP4 with EP4 per replica. The trade is fidelity: mean KL to the original is 0.0522 on neutral text, against a noise floor of 0.0082, while GSM8K-200 reaches 98.5% and HumanEval 97.0%. Download the repo with hf download → Build the Docker image and the kernels → Start serve.sh with an API key → vLLM splits the model across two GPUs → Some experts wait in pinned host RAM → Clients call the API on port 8000 ### Good to know Output speed, agentic replay, one stream | Label | Value | | --- | --- | | 2x default | 224.1 tok/s | | 2x CPU runtime | 261.8 tok/s | | Previous release | 176 tok/s | - Speed: on the agentic replay benchmark, the default setup is 28% quicker at decoding than the last V4.1-Flash release. - Context: the default setup offers a 1M window with room for 1.07M tokens in an fp8 KV cache. - Hardware: the CPU runtime needs an x86\_64 CPU with AVX-512, many cores and many memory channels. - License: mit - Cost to run: no paid service named. You supply the GPUs and pick your own API key value. - Data it reaches: it downloads files from Hugging Face, and the server answers requests on port 8000 of your machine. - Not for: setups below the listed GPU, RAM and disk needs. ## README ````markdown # DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 Serves a 3-bit compressed copy of DeepSeek-V4.1-Flash-1017 on two 96 GB RTX PRO 6000 Blackwell cards with vLLM, for developers who host a large coding and agent model on their own hardware. Made by diffbot on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a 3-bit compressed copy of DeepSeek-V4.1-Flash-1017, a text generation model with 219B parameters. The author calibrated the EXL3 3-bit routed experts themselves. vLLM runs it across a pair of RTX PRO 6000 Blackwell cards, and about a third of the experts wait in pinned host RAM. Who it suits: It suits engineers and teams who host a large coding or agent model on their own GPU machines and want an OpenAI-compatible API in front of it. ```table Industry | Who uses it | How they use it | Example Software development | Engineers with two 96 GB GPUs | Serve the model for coding agents | Point an agent at port 8000 with the API key you set AI infrastructure | Teams with an 8-GPU node | Run one copy on each half of the node, keeping all experts in VRAM | Send requests to port 8000 or 8001 Model research | People who study quantization | Compare the pack against the original model | Read its mean KL and benchmark tables ``` Anyone without the GPUs, memory and disk listed below can skip it. Where to use it: - Machines with Docker, the NVIDIA container toolkit and two Blackwell sm_120 cards - vLLM on a pair of RTX PRO 6000 cards, split with tensor parallelism of 2 - An 8x RTX PRO 6000 node, run as two replicas of four GPUs each - Clients of an OpenAI-compatible API served on port 8000 - Windows, macOS or Linux with Python, only to fetch the files ```table Use case | Who | What they get back Coding agent backend | Developers | Fast replies in multi-turn work with tool calls Long context work | Developers | A 1M-token window with an fp8 KV cache Quality checking | Researchers | Published KL, GSM8K-200 and HumanEval numbers for the pack ``` How to set it up: What you need first: - Two Blackwell sm_120 GPUs with 96 GB each - A CUDA 13 driver on the host - Docker with the NVIDIA container toolkit - Around 260 GiB of spare host RAM - 340 GB of disk 1. Install the Hugging Face tools with pip. 2. Download the repository into a local folder. 3. Open the recipe folder and build the serving image. 4. Build the kernels, which takes about 4 minutes. 5. Start the server with an API key you choose. ```bash pip install -U huggingface_hub ``` ```bash hf download diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 --local-dir ~/models/DeepSeek-V4.1-Flash-1017-EXL3-3bpw cd ~/models/DeepSeek-V4.1-Flash-1017-EXL3-3bpw/recipe bash docker/build.sh # serving image dsv41-flash-1017-exl3-3bpw-sm120 bash kernels/build.sh # the xmoe, xmoe_do, xmoe_do_i8, fp8ar and rowtopk kernels, about 4 minutes ``` Put your own value in API_KEY first. An x86_64 CPU with AVX-512 support (VNNI, VBMI, VBMI2), plus many cores and memory channels, lets you add the optional CPU runtime. Build it and run the host check, then switch it on. ```bash bash cpuexp/build.sh && bash cpuexp/check_host.sh ``` Put your own value in API_KEY first. On an 8x RTX PRO 6000 node, download the whole repo the same way, including the drafter-code-k7 folder, then run these. ```bash bash docker/build.sh # the same image, now with the chunked-indexer patch bash kernels/build.sh 8x # xmoe_p8k, xmoe_do_i8, fp8ar, fp8arg and rowtopk ``` Put your own value in API_KEY first. How to use it: After serve.sh starts, any client can reach its OpenAI-compatible API on port 8000 and ask for the model named deepseek-v4.1-flash. The scripts below check the running server. ```bash python3 bench/needle-test.py 200 && python3 bench/test-toolcall.py && python3 bench/test-image.py ``` Put your own value in API_KEY first. For a single stream of mostly code on two cards, you can switch on the code-tuned DSpark head with k set to 7. Put your own value in API_KEY first. ```table Tool | What it does serve/serve.sh | Starts the server on two cards, with every setting documented inside serve/serve-8x.sh | Starts two replicas on an 8-GPU node bench/needle-test.py | Runs the needle test against the server bench/test-toolcall.py | Runs the tool call test bench/test-image.py | Runs the image test ``` Example: You start the server with the default command, then send a chat request to port 8000 with your API key and the model name deepseek-v4.1-flash. The quantized model's reply comes back through the OpenAI-compatible API. For newcomers and experts: This is a huge model, with about 437 GB of weights, so it needs a GPU server and not a laptop. A first small step is to download the files and read the recipe README, which explains the stack. Then run the build and start commands one by one. Experts will want the details. The author quantized the routed experts to 3 bits with EXL3, using their own calibration and a method credited to coolbho3k. vLLM splits the model over two cards with tensor parallelism of 2. About a third of the experts stay in pinned host RAM, and custom sm_120 kernels or a CPU runtime that needs AVX-512, which is optional, serve them. The 8x profile uses TP4 with EP4 per replica. The trade is fidelity: mean KL to the original is 0.0522 on neutral text, against a noise floor of 0.0082, while GSM8K-200 reaches 98.5% and HumanEval 97.0%. ```flow Download the repo with hf download Build the Docker image and the kernels Start serve.sh with an API key vLLM splits the model across two GPUs Some experts wait in pinned host RAM Clients call the API on port 8000 ``` Good to know: ```chart Title: Output speed, agentic replay, one stream 2x default: 224.1 tok/s 2x CPU runtime: 261.8 tok/s Previous release: 176 tok/s ``` - Speed: on the agentic replay benchmark, the default setup is 28% quicker at decoding than the last V4.1-Flash release. - Context: the default setup offers a 1M window with room for 1.07M tokens in an fp8 KV cache. - Hardware: the CPU runtime needs an x86_64 CPU with AVX-512, many cores and many memory channels. - License: mit - Cost to run: no paid service named. You supply the GPUs and pick your own API key value. - Data it reaches: it downloads files from Hugging Face, and the server answers requests on port 8000 of your machine. - Not for: setups below the listed GPU, RAM and disk needs. ## On Hugging Face, 13 November 2026 - Task: text generation - Parameters: 219B - Weights: about 437 GB in I16 - Formats: Safetensors - Downloads in the last 30 days: 2,514 - Likes: 42 - License: mit - Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017 - Quantized versions: 1 - Tags: safetensors, deepseek_v41, exl3, exllamav3, vllm, deepseek, deepseek-v4.1-flash, deepseek-v4.1-flash-1017, mixture-of-experts, 3-bit, rtx-pro-6000, blackwell ## Links - Model: https://huggingface.co/diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 - Files: https://huggingface.co/diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 on Hugging Face](https://huggingface.co/diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 (this page) | 2,514 | 42 | 219B | mit | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017-EXL3-franken-v100](https://mindket.com/p/91bd8bd6-2fba-4896-ae7f-5c8ff314c941) | 313 | 7 | — | mit | 2026-10-26 | | [DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw](https://mindket.com/p/e2aa9049-7d40-46f6-b187-57f9ffaaccf1) | 8,113 | 46 | 105B | mit | 2026-10-20 | | [DeepSeek-V4.1-Flash-1017 TrellisMX K2 Mix](https://mindket.com/p/07ee61ca-5fd5-44fb-a019-f885468d68a4) | 338 | 12 | 115B | mit | 2026-10-20 | | [GLM-5.3-Flash EXL3 3.05bpw](https://mindket.com/p/783914ac-53d0-4c6b-adb4-69db026aa354) | 0 | 2 | — | mit | 2026-10-12 | | [Qwen3.8-27B-NVFP4-DGX-SPARK](https://mindket.com/p/a7666325-428a-4e43-b11c-5fec915dc5df) | 41,306 | 1,127 | 18.2B | apache-2.0 | 2026-10-28 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-09 - First published: 2026-09-11 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000 on Hugging Face](https://huggingface.co/diffbot/DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e) - [Its page on Mindket](https://mindket.com/p/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e)