DeepSeek-V4.1-Flash-1017-exl3

DeepSeek-V4.1-Flash-1017-exl3

Runs a compressed DeepSeek-V4.1-Flash-1017 in ExLlamaV3 at 1.68, 2.25 or 2.68 bits per weight. For people who run large models on their own hardware and want less memory use.

Made by turboderp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: DeepSeek-V4.1-Flash-1017-exl3 holds compressed copies of the DeepSeek-V4.1-Flash-1017 model in the EXL3 format. Each copy stores the weights with fewer bits, so the model needs less memory. The card lists three sizes, at 1.68, 2.25 and 2.68 bits per weight.

Who it suits: People who run large language models on their own hardware with ExLlamaV3 and want a smaller copy of this DeepSeek model.

Industry | Who uses it | How they use it | Example
Software | Local model users | Run a DeepSeek model in less memory | Pick the 1.68 bits per weight copy to save space
AI infrastructure | Engineers | Self-host the model with ExLlamaV3 | Serve the 2.25 bits per weight copy
Research | Quantization researchers | Compare the three sizes on benchmark scores | Read the AIME-25 and GPQA-D results for each

If you do not use ExLlamaV3, you can skip it, since the card names no other runtime.

Where to use it:

• ExLlamaV3 version 1.6.6, or its dev branch
• Your own computer with Python, on Windows, macOS or Linux, for the download

Use case | Who | What they get back
Math problems | Students and researchers | Answers from a model the card scores on AIME-25
Science questions | Researchers | Answers from a model the card scores on GPQA-D
Memory-limited setups | Local model users | A smaller copy of the base model

How to set it up:

• Python with pip
• ExLlamaV3 version 1.6.6, or its dev branch
• Enough memory for the copy you pick, since the card gives bits per weight but no file sizes

1. Install the Hugging Face hub tool.
2. Download the model files.
3. Load them in ExLlamaV3 and choose one of the three sizes.

pip install -U huggingface_hub
hf download turboderp/DeepSeek-V4.1-Flash-1017-exl3

How to use it: Load the copy you chose in ExLlamaV3, then prompt it as you would the original DeepSeek model. The card shows no command, so follow ExLlamaV3's own steps for loading an EXL3 model.

Example: You load the 2.25 bits per weight copy in ExLlamaV3 and send it a competition math question, since the card scores that copy on AIME-25. This only illustrates the use, because the card shows no sample reply.

For newcomers and experts: If you are new, a quant is a shrunk copy of a model. Fewer bits per weight means less memory, and the card's scores show what each size keeps. Start with the 2.68 bits per weight copy, which has the most bits of the three.

For experts, the card calls these SC quants and plots KL divergence and perplexity against VRAM in two images. It says more quants and more benchmarks will follow. A fixed baseline score on AIME-25 was assumed, because the author found no source.

The DeepSeek-V4.1-Flash-1017 base model
EXL3 compresses its weights to fewer bits
You pick the 1.68, 2.25 or 2.68 size
ExLlamaV3 loads it for chat

Good to know: The card scores each copy on AIME-25 and GPQA-D. The 1.68 copy gets 96.7% on AIME-25, and the other two get 100%. On GPQA-D the copies range from 89.9% to 90.4%. The baseline GPQA-D score of 90.2% comes from OpenRouter, and the AIME-25 baseline of 100% is an assumption.

Title: GPQA-D score by size
1.68 bits per weight: 90.4%
2.25 bits per weight: 90.4%
2.68 bits per weight: 89.9%
Baseline: 90.2%

If you see corrupted output on short prompts, the card says it is likely a failure mode of the model itself.

• License: MIT, as the Hugging Face record lists it.
• Cost to run: no API key or account is named. The card gives no hardware needs.
• Data it reaches: downloading reaches Hugging Face. The card does not describe the model reaching files or the web.
• Not for: older ExLlamaV3 versions, since the card says it requires version 1.6.6 or the dev branch.

On Hugging Face, 13 November 2026

• Task: image text to text
• Downloads in the last 30 days: 581
• Likes: 30
• License: mit
• Based on: deepseek-ai/DeepSeek-V4.1-Flash-1017
• Tags: exl3, image-text-to-text

Links

• Model: https://huggingface.co/turboderp/DeepSeek-V4.1-Flash-1017-exl3
• Files: https://huggingface.co/turboderp/DeepSeek-V4.1-Flash-1017-exl3/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download turboderp/DeepSeek-V4.1-Flash-1017-exl3

Then follow the usage in its model card. Nothing in this kit runs by itself.