DeepSeek-V4.1-Flash-1017-EXL3-franken-v100

DeepSeek-V4.1-Flash-1017-EXL3-franken-v100

Runs DeepSeek V4.1 Flash at its full 1M-token context on eight Tesla V100 32 GB cards, using an EXL3 expert pack and an SM70 vLLM build. For owners of such servers.

Made by philbert440 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is an EXL3 expert pack of DeepSeek-V4.1-Flash-1017, sized for eight Tesla V100-SXM2-32GB cards linked by NVLink. The 1Cat-vLLM engine, ported to SM70, runs it with the model's full 1,048,576-token window and DSpark speculative decoding. The routed experts average about 2.8 bits per weight.

Who it suits: People who own an eight-card V100 server and want long-context DeepSeek V4.1 Flash on it. It fits those who are comfortable installing a custom vLLM build.

Industry | Who uses it | How they use it | Example
Self-hosted AI | Owners of eight-card V100 servers | Serve a large model with a 1M-token context | Run chat and coding requests against a local server
Agentic coding | Developers on that hardware | Use the model for long coding sessions | The author reports about 80 tok/s decode in single-stream sessions
Vision workflows | Teams sending images with requests | Pass images as part of a chat request | A screenshot inside a tool result was read in 1.2 s, the card says

Anyone without that hardware, or who wants a one-click install, can skip it.

Where to use it:

• A server with eight Tesla V100-SXM2-32GB cards, 256 GB of memory in total.
• The author's SM70 build of vLLM, with a launcher script named serve_ds41.sh.
• OpenAI-style chat requests, including image_url content parts.
• Windows, macOS or Linux: only the file download runs there with Python.

Use case | Who | What they get back
Long-context chat | Developers with the server | Replies from a context window of 1,048,576 tokens
Coding sessions | Developers with the server | About 64.9 tok/s decode on code prompts, one stream, in the card's test
Image questions | Developers with the server | Text read back from an image, up to 64 images per prompt

How to set it up: What you need first:

• Eight Tesla V100-SXM2-32GB cards with NVLink.
• Python with pip, to fetch the files.
• The author's SM70 vLLM build, which the card says comes with a prebuilt wheel.

1. Install the Hugging Face hub package with pip.
2. Download the pack with the hf command.
3. Install the author's SM70 vLLM build.
4. Start the server with the launcher script serve_ds41.sh from that build's v100 folder.

pip install -U huggingface_hub
hf download philbert440/DeepSeek-V4.1-Flash-1017-EXL3-franken-v100

How to use it: Once the server runs, you send OpenAI-style chat requests to it. To include images, add image_url content parts, up to 64 images per prompt, also inside tool results. Each image takes at most 1,024 tokens of the prompt. Each engine addition sits behind a VLLM_DS41_ switch.

Title: Parts of the setup the card names
Tool | What it does
serve_ds41.sh | The live launcher script in the author's engine build
VLLM_DS41_ switches | Gate each engine addition, such as the faster prefill kernels
image_url parts | Carry images in a request, up to 64 per prompt
DSpark | Speculative decoding, tested with k=5 on one and two streams

Example: You send a coding prompt to your own server and attach a screenshot of an error as an image_url part. The image counts as up to 1,024 prompt tokens. The card's test of code prompts on one stream decoded at about 64.9 tok/s.

For newcomers and experts: This is an advanced setup, not a download-and-run model. It is a compressed copy of a large model plus serving software for one specific eight-card server. A good first step is to fetch the files with the two commands and read the card's recipe to check that your cards match.

Nothing was re-quantized. The pack mixes experts picked one by one from two public EXL3 packs. That choice leaves room on each 32 GB card for the drafter and the long-context cache as well. The dense, attention, Engram and MTP tensors keep the official FP8, BF16 and MXFP4 weights. A fix for fp16 overflow in the drafter's hidden-state taps removed all-zero drafts at the start of each chat.

Fetch the pack and the SM70 vLLM build
Launch serve_ds41.sh on eight V100 cards
Decode with DSpark speculative decoding
Return replies, with images as image_url parts

Good to know: These speeds are the card's own numbers for decode on one stream.

Title: Decode speed on one stream, DSpark k=5
Code: 64.9 tok/s
Structured JSON: 65.9 tok/s
Chat: 35.0 tok/s
Prose: 34.5 tok/s

• Quality check: the card reports a KL of 0.0358 against an fp32 reference, with 95.79 % top-1 agreement over 3,134 positions.
• Context: 1,048,576 tokens, with a KV pool of 1,055,580 tokens.
• Speed at length: about 51 tok/s decode at 8K and 52.6 tok/s at 128K.
• License: mit
• Cost to run: no API key, account or paid service named, but it needs eight V100 32 GB cards.
• Data it reaches: files you download from Hugging Face, and the prompts and images you send to your own server.
• Not for: other GPU setups, since the pack is sized for eight V100 32 GB cards.

On Hugging Face, 13 November 2026

• Library: vllm
• Formats: Safetensors
• Downloads in the last 30 days: 313
• Likes: 7
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Tags: vllm, safetensors, deepseek_v41, exl3, exllamav3, quantized, deepseek, v100, sm70, 1cat-vllm, speculative-decoding, vision

Links

• Model: https://huggingface.co/philbert440/DeepSeek-V4.1-Flash-1017-EXL3-franken-v100
• Files: https://huggingface.co/philbert440/DeepSeek-V4.1-Flash-1017-EXL3-franken-v100/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download philbert440/DeepSeek-V4.1-Flash-1017-EXL3-franken-v100

Then follow the usage in its model card. Nothing in this kit runs by itself.