DeepSeek V4.1 Flash GGUF for DwarfStar

DeepSeek V4.1 Flash GGUF for DwarfStar

Gives Mac users calibrated 2-bit and 4-bit GGUF files of DeepSeek V4.1 Flash, plus a separate vision encoder, for DwarfStar's Metal engine with SSD streaming. Not interchangeable with other DeepSeek weights.

Made by antirez on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds calibrated GGUF weights of DeepSeek V4.1 Flash for the Metal engine in DwarfStar, a project by antirez. There are three download targets: a Q2 build, a Q4 build and a small separate vision encoder. The card warns that these files cannot stand in for the DeepSeek V4 Flash weights or for the DSpark drafter that goes with them.

Who it suits: It suits Mac owners with a lot of memory and a fast SSD who want to run a huge DeepSeek model locally through DwarfStar.

Industry | Who uses it | How they use it | Example
Local AI | Owners of a 128 GB Mac | Run the Q2 build with SSD streaming | Start ds4 on the Q2 file with a 32768-token context
Software | Developers building agents | Use the ds4-agent and ds4-server programs with the same options | Serve the model locally to an agent
Research | Quantization enthusiasts | Compare a Q2 build with a Q4 build made from one imatrix | Read how routed experts are quantized at each level

You can skip it if you do not have a Mac with Metal, or if you plan to use another runtime, since the card covers DwarfStar only.

Where to use it:

• DwarfStar's ds4, ds4-agent and ds4-server programs on a Mac
• A single 128 GB Mac with SSD streaming for Q2
• Two 128 GB Macs with RDMA for resident Q2, or one larger Mac
• Windows, macOS and Linux with Python, only to download with Hugging Face's client

Use case | Who | What they get back
Local chat | Mac owners | Replies from a huge model on one Mac with SSD streaming
Local server | Developers | A served model through ds4-server
Image input | Users with the vision file | A model that can take images

How to set it up: You need these first:

• A DwarfStar checkout with V4.1 support
• A Mac with Metal and a fast local SSD
• Disk space for 340.60 GiB for Q2 or 482.98 GiB for Q4, plus 37 GiB spare while Q4 joins its parts

1. Get a DwarfStar checkout that supports V4.1.
2. Run its download script with the Q2 target.
3. Start ds4 on the downloaded file with SSD streaming and your context size.
4. For Q4 or images, run the download script again with those targets.

./download_model.sh ds41f-q2
./ds4 -m gguf/DeepSeek-V4.1-Flash-Q2.gguf --ssd-streaming --ctx 32768

./download_model.sh ds41f-q4
./download_model.sh ds41f-vision

How to use it: You run ds4 on the Q2 or Q4 file and chat with it. For images, you add the vision file to the same command with the vision flag. The ds4-agent and ds4-server programs take identical model and memory settings.

Tool | What it does
download_model.sh | Downloads a target, joining and checking the Q4 parts
ds4 | Runs the model from a GGUF file
--ssd-streaming | Reads data from the SSD instead of holding everything in memory
--ctx | Sets the context length in tokens
--vision | Adds the vision encoder file to the language model command
ds4-agent and ds4-server | Run an agent or a server with the same model and memory options

Example: You run the Q2 download, then start ds4 on the Q2 file with SSD streaming and a context of 32768. The model loads from your SSD on a single 128 GB Mac. You then chat with it in the terminal.

For newcomers and experts: This is a huge model that only a high-memory Mac can run, and only with the DwarfStar engine. A good first step is the two Q2 commands above, after you check that your Mac and SSD fit the numbers.

For readers who know the field, every language file carries 188.83 GiB of FP8 Engram tables stored natively, and DwarfStar pulls the rows it needs straight from disk. Q2 uses IQ2_XXS for gate and up experts and Q2_K for down experts. In the Q4 build, every routed expert is stored as Q4_K. Both keep Q8 attention projections and share one imatrix from 8,192-token activations.

Get a DwarfStar checkout with V4.1 support
Run download_model.sh for a target
Start ds4 with the GGUF file
Rows of the Engram tables stream from SSD
Chat in the terminal or serve it

Good to know:

• Compatibility: do not swap these files in for DeepSeek V4 Flash weights, or use them with its DSpark drafter.
• Storage: keep the file on a fast local SSD, since Engram rows are read from it during use.
• Sizes: the main-weight sizes leave out context and runtime buffers.
• Q4 files: Q4 is stored in two parts that the downloader joins, and the parts alone do not run.
• Memory: Q4 needs SSD streaming on smaller Macs, and a 512 GB Mac is the target for resident use.
• Counts: Hugging Face lists 755B parameters and about 377 GB at 4-bit, but the card gives its own file sizes.

Title: File size by download target
ds41f-q2: 340.60 GiB
ds41f-q4: 482.98 GiB
ds41f-vision: 0.90 GiB

• License: MIT, with the upstream MIT license included.
• Cost to run: none named. You need a high-memory Mac and a fast SSD.
• Data it reaches: reads the model files on your SSD, and the download script fetches them. The text available here names no other access.
• Not for: other runtimes, or mixing with DeepSeek V4 Flash weights. The two Q4 parts are not runnable on their own.

On Hugging Face, 13 November 2026

• Parameters: 755B
• Weights: about 377 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 918,185
• Likes: 94
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash
• Tags: gguf, deepseek, metal, imatrix

Links

• Model: https://huggingface.co/antirez/deepseek-v4.1-flash-gguf
• Files: https://huggingface.co/antirez/deepseek-v4.1-flash-gguf/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download antirez/deepseek-v4.1-flash-gguf

Then follow the usage in its model card. Nothing in this kit runs by itself.