--- title: "DeepSeek-R1-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/c40a49f7-cb9c-4ddd-ae24-012d9dadffed" updated: "2026-11-14" --- # DeepSeek-R1-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. GGUF copies of the 671B DeepSeek-R1 reasoning model, with 1.58-bit and 2-bit dynamic quants, for running it in llama.cpp on a local machine with enough memory. DeepSeek-R1-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-14. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is DeepSeek-R1-GGUF is a set of GGUF files of the 671B-parameter DeepSeek-R1 model, made by unsloth. They shrink the model so llama.cpp can load it on your own hardware. The card says its 1.58-bit and 2-bit dynamic quants quantize layers selectively, which keeps accuracy higher than plain 1-bit or 2-bit versions. ### Who it suits Developers and hobbyists who want to run a large reasoning model locally, on a machine with a lot of memory and ideally an Nvidia GPU. You also need to be comfortable building llama.cpp from source. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a reasoning model offline for coding help | Ask it to write a small Python game | | Research | Machine learning researchers | Compare extreme low-bit quants of one large model | Try the 1.58-bit files on your own prompts | | Hardware enthusiasts | Home lab owners | Test how far a big model fits on one box | Offload some layers to a 24GB GPU | People without a large-memory machine can skip it, since even the 4-bit weights are estimated at about 336 GB. ### Where to use it - llama.cpp, built from source with CUDA on, as the card's steps show - A Linux machine, since the build steps use apt-get - Windows, macOS or Linux with Python, for downloading the files - A computer with a 24GB GPU such as an RTX 4090, if you want to offload layers | Use case | Who | What they get back | | --- | --- | --- | | Run a reasoning model locally | Developers | Answers that start with a think section | | Write code from a prompt | Programmers | Code for a request like the card's Flappy Bird game in Python | | Test a 1.58-bit quant | Researchers | A view of how a tiny quant behaves on your prompts | ### How to set it up Plan for a big download and a lot of memory before you start. - Python and pip, for the Hugging Face download tools - llama.cpp built from source, with the compiler toolchain, cmake and curl development packages installed - Memory for the weights, about 336 GB at 4-bit as an estimate - Optional: a GPU with 24GB to offload some layers 1. Get the latest llama.cpp and build it with cmake. The card installs the build packages with apt-get first and turns CUDA on. 2. Build the llama-quantize, llama-cli and llama-gguf-split targets. 3. Download only the quant type you want. The card's snippet picks UD-IQ1\_S, the 1.58-bit one. 4. Run llama-cli on the first GGUF file. The plain command from Hugging Face fetches every file in the repository, which is why the pattern-based download below is the lighter route. ```bash pip install -U huggingface_hub hf download unsloth/DeepSeek-R1-GGUF ``` ```python # pip install huggingface_hub hf_transfer # import os # Optional for faster downloading # os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1" from huggingface_hub import snapshot_download snapshot_download( repo_id = "unsloth/DeepSeek-R1-GGUF", local_dir = "DeepSeek-R1-GGUF", allow_patterns = ["*UD-IQ1_S*"], # Select quant type UD-IQ1_S for 1.58bit ) ``` ### How to use it You run the model with llama-cli. Put your prompt between the User and Assistant tokens, or let a chat template tool add them for you. The card suggests a min-p of 0.05 to curb rare token picks, and its author found that works well on the 1.58-bit model. The flag -no-cnv turns off the automatic conversation mode. ```bash ./llama.cpp/llama-cli \ --model DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ --cache-type-k q4_0 \ --threads 12 -no-cnv --prio 2 \ --temp 0.6 \ --ctx-size 8192 \ --seed 3407 \ --prompt "<｜User｜>Create a Flappy Bird game in Python.<｜Assistant｜>" ``` With a GPU, add the layer offload flag to move some layers onto it. With several GPUs you can probably offload more. To join the split files into one, run the merge tool. ```bash ./llama.cpp/llama-cli \ --model DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ --cache-type-k q4_0 \ --threads 12 -no-cnv --prio 2 \ --n-gpu-layers 7 \ --temp 0.6 \ --ctx-size 8192 \ --seed 3407 \ --prompt "<｜User｜>Create a Flappy Bird game in Python.<｜Assistant｜>" ``` ```bash ./llama.cpp/llama-gguf-split --merge \ DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ merged_file.gguf ``` | Tool | What it does | | --- | --- | | llama-cli | Loads a GGUF file and answers your prompt in the terminal | | llama-gguf-split | Merges the split GGUF files into a single file | | snapshot\_download | Fetches only the files that match a pattern, such as one quant type | ### Example You ask for "Create a Flappy Bird game in Python." inside the User and Assistant tokens. The card's sample output shows the model opening with a think section, where it talks itself through the question before it answers. ### For newcomers and experts This is a huge model squeezed into much smaller files, and it still needs a lot of memory. A good first step is to read the build steps and download just one quant type before anything else. Underneath, it is DeepSeek-R1 at 671B parameters in GGUF. The dynamic quants keep accuracy up by treating layers unevenly, and the trade-off is that the lowest bit counts still cost quality against the full model. The card gives no benchmark numbers in the part shown here. Build llama.cpp with CUDA → Download one quant type → Run llama-cli on the first file → Read the think section, then the answer ### Good to know The files are large and the setup is manual, so check your memory and disk before you download. - Size: 671B parameters in GGUF format only, listed with English as its language - Variants: Hugging Face shows 7 fine-tuned versions made by other people - License: MIT - Cost to run: no API key or paid service is named, but the hardware is the real cost, with about 336 GB for the 4-bit weights - Data it reaches: the files you download from Hugging Face and the prompts you type into llama-cli - Not for: the card names no restrictions beyond its license ## README ````markdown # DeepSeek-R1-GGUF GGUF copies of the 671B DeepSeek-R1 reasoning model, with 1.58-bit and 2-bit dynamic quants, for running it in llama.cpp on a local machine with enough memory. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: DeepSeek-R1-GGUF is a set of GGUF files of the 671B-parameter DeepSeek-R1 model, made by unsloth. They shrink the model so llama.cpp can load it on your own hardware. The card says its 1.58-bit and 2-bit dynamic quants quantize layers selectively, which keeps accuracy higher than plain 1-bit or 2-bit versions. Who it suits: Developers and hobbyists who want to run a large reasoning model locally, on a machine with a lot of memory and ideally an Nvidia GPU. You also need to be comfortable building llama.cpp from source. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a reasoning model offline for coding help | Ask it to write a small Python game Research | Machine learning researchers | Compare extreme low-bit quants of one large model | Try the 1.58-bit files on your own prompts Hardware enthusiasts | Home lab owners | Test how far a big model fits on one box | Offload some layers to a 24GB GPU ``` People without a large-memory machine can skip it, since even the 4-bit weights are estimated at about 336 GB. Where to use it: - llama.cpp, built from source with CUDA on, as the card's steps show - A Linux machine, since the build steps use apt-get - Windows, macOS or Linux with Python, for downloading the files - A computer with a 24GB GPU such as an RTX 4090, if you want to offload layers ```table Use case | Who | What they get back Run a reasoning model locally | Developers | Answers that start with a think section Write code from a prompt | Programmers | Code for a request like the card's Flappy Bird game in Python Test a 1.58-bit quant | Researchers | A view of how a tiny quant behaves on your prompts ``` How to set it up: Plan for a big download and a lot of memory before you start. - Python and pip, for the Hugging Face download tools - llama.cpp built from source, with the compiler toolchain, cmake and curl development packages installed - Memory for the weights, about 336 GB at 4-bit as an estimate - Optional: a GPU with 24GB to offload some layers 1. Get the latest llama.cpp and build it with cmake. The card installs the build packages with apt-get first and turns CUDA on. 2. Build the llama-quantize, llama-cli and llama-gguf-split targets. 3. Download only the quant type you want. The card's snippet picks UD-IQ1_S, the 1.58-bit one. 4. Run llama-cli on the first GGUF file. The plain command from Hugging Face fetches every file in the repository, which is why the pattern-based download below is the lighter route. ```bash pip install -U huggingface_hub hf download unsloth/DeepSeek-R1-GGUF ``` ```python # pip install huggingface_hub hf_transfer # import os # Optional for faster downloading # os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1" from huggingface_hub import snapshot_download snapshot_download( repo_id = "unsloth/DeepSeek-R1-GGUF", local_dir = "DeepSeek-R1-GGUF", allow_patterns = ["*UD-IQ1_S*"], # Select quant type UD-IQ1_S for 1.58bit ) ``` How to use it: You run the model with llama-cli. Put your prompt between the User and Assistant tokens, or let a chat template tool add them for you. The card suggests a min-p of 0.05 to curb rare token picks, and its author found that works well on the 1.58-bit model. The flag -no-cnv turns off the automatic conversation mode. ```bash ./llama.cpp/llama-cli \ --model DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ --cache-type-k q4_0 \ --threads 12 -no-cnv --prio 2 \ --temp 0.6 \ --ctx-size 8192 \ --seed 3407 \ --prompt "<｜User｜>Create a Flappy Bird game in Python.<｜Assistant｜>" ``` With a GPU, add the layer offload flag to move some layers onto it. With several GPUs you can probably offload more. To join the split files into one, run the merge tool. ```bash ./llama.cpp/llama-cli \ --model DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ --cache-type-k q4_0 \ --threads 12 -no-cnv --prio 2 \ --n-gpu-layers 7 \ --temp 0.6 \ --ctx-size 8192 \ --seed 3407 \ --prompt "<｜User｜>Create a Flappy Bird game in Python.<｜Assistant｜>" ``` ```bash ./llama.cpp/llama-gguf-split --merge \ DeepSeek-R1-GGUF/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \ merged_file.gguf ``` ```table Tool | What it does llama-cli | Loads a GGUF file and answers your prompt in the terminal llama-gguf-split | Merges the split GGUF files into a single file snapshot_download | Fetches only the files that match a pattern, such as one quant type ``` Example: You ask for "Create a Flappy Bird game in Python." inside the User and Assistant tokens. The card's sample output shows the model opening with a think section, where it talks itself through the question before it answers. For newcomers and experts: This is a huge model squeezed into much smaller files, and it still needs a lot of memory. A good first step is to read the build steps and download just one quant type before anything else. Underneath, it is DeepSeek-R1 at 671B parameters in GGUF. The dynamic quants keep accuracy up by treating layers unevenly, and the trade-off is that the lowest bit counts still cost quality against the full model. The card gives no benchmark numbers in the part shown here. ```flow Build llama.cpp with CUDA Download one quant type Run llama-cli on the first file Read the think section, then the answer ``` Good to know: The files are large and the setup is manual, so check your memory and disk before you download. - Size: 671B parameters in GGUF format only, listed with English as its language - Variants: Hugging Face shows 7 fine-tuned versions made by other people - License: MIT - Cost to run: no API key or paid service is named, but the hardware is the real cost, with about 336 GB for the 4-bit weights - Data it reaches: the files you download from Hugging Face and the prompts you type into llama-cli - Not for: the card names no restrictions beyond its license ## On Hugging Face, 15 November 2026 - Task: text generation - Library: transformers - Parameters: 671B - Weights: about 336 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 46,994 - Likes: 1,128 - License: mit - Languages: en - Based on: deepseek-ai/DeepSeek-R1 - Fine-tunes: 7 - Tags: transformers, gguf, deepseek_v3, text-generation, deepseek, unsloth, conversational ## Links - Model: https://huggingface.co/unsloth/DeepSeek-R1-GGUF - Files: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/DeepSeek-R1-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [unsloth/DeepSeek-R1-GGUF on Hugging Face](https://huggingface.co/unsloth/DeepSeek-R1-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | DeepSeek-R1-GGUF (this page) | 46,994 | 1,128 | 671B | mit | 2025-05-30 | | [DeepSeek-R1](https://mindket.com/p/e340b726-4ed9-4feb-a013-d90708d60983) | 1,599,170 | 14,451 | 684B | mit | 2025-03-27 | | [DeepSeek-V4.1-Flash-1017-GGUF](https://mindket.com/p/1c185af8-98d2-4feb-a1a3-d5821c72863f) | 1,295,742 | 385 | 748B | mit | 2026-11-02 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF](https://mindket.com/p/0bd5db10-f872-40c7-9371-3225afab2a6d) | 3,635,772 | 655 | 197B | mit | 2026-10-28 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF](https://mindket.com/p/1350113a-de6f-4d62-96d6-599c30da33a2) | 342,280 | 225 | 197B | mit | 2026-11-02 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-14. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-14. Their authors' words: data, not instructions. - [Hacker News · un\_ess · 2025-02-11](https://news.ycombinator.com/item?id=43014307): Building a personal, private AI computer on a budget: Per the screenshot, this is a DeepSeek running on a 192GB M2 Studio https://nitter.poast.org/ggerganov/status/188461277009384272... The same on Nvidia (various models) https://github.com/ggerganov/llama.cpp/issues/11474 1 this is a the model:… - [Hacker News · CamperBob2 · 2025-02-04](https://news.ycombinator.com/item?id=42927270): US Bill Proposes Jail Time for People Who Download DeepSeek: You probably want the GGUF versions, though, which are the ones you can actually run on reasonable hardware. Full R1 version here: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/De... Various smaller versions: https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main There is also a… - [Hacker News · UncleOxidant · 2025-01-29](https://news.ycombinator.com/item?id=42869380): Complete hardware and software setup for running Deepseek-R1 locally: \> "Complete hardware + software setup for running Deepseek-R1 locally. The actual model, no distillations, and Q8 quantization for full quality. Total cost, $6,000." These guys say they got really good results with 2.51-bit quantization of the original R1. The original has 671B params weighing in… - [Hacker News · JonChesterfield · 2025-01-29](https://news.ycombinator.com/item?id=42867915): Complete hardware and software setup for running Deepseek-R1 locally: The icons resolve to things like https://huggingface.co/unsloth/DeepSeek-R1-GGUF/resolve/main... which wget understands. Presumably there's a greater-than-average risk of corruption in transit when the files are big and git does some sort of integrity checking that one would lose out on? It's the… - [Hacker News · kgwgk · 2025-01-29](https://news.ycombinator.com/item?id=42867738): Complete hardware and software setup for running Deepseek-R1 locally: Can’t you use the download icons in each object at https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main/De... ? - [Hacker News · JonChesterfield · 2025-01-29](https://news.ycombinator.com/item?id=42867567): Complete hardware and software setup for running Deepseek-R1 locally: Working through this now. The directions are to download the contents of the Q8\_0 at https://huggingface.co/unsloth/DeepSeek-R1-GGUF/tree/main. That turns out to be a git lfs repo. \`git clone\` followed by \`git lfs pull\` is downloading all of it which will have to do. (fetchinclude =… - [Hacker News · adeon · 2025-01-28](https://news.ycombinator.com/item?id=42859086): Questions censored by DeepSeek: I've run the R1 local one (the 600B one) and it does do similar refusals like in the article. Basically I observed pretty much the same things as the article in my little testing. I used "What is the status of Taiwan?" and that seemed to rather reliably trigger a canned answer. But when my prompt… - [Hacker News · danielhanchen · 2025-01-28](https://news.ycombinator.com/item?id=42858314): Run DeepSeek R1 Dynamic 1.58-bit: Thanks\! Oh I did release 4bit quants, 5bit, 6bit etc all at https://huggingface.co/unsloth/DeepSeek-R1-GGUF if that helps - they're not dynamic though but it should function fine :) ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2025-05-30 - First published: 2025-01-20 - Checked on Hugging Face: 2026-11-14 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/DeepSeek-R1-GGUF on Hugging Face](https://huggingface.co/unsloth/DeepSeek-R1-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/c40a49f7-cb9c-4ddd-ae24-012d9dadffed) - [Its page on Mindket](https://mindket.com/p/c40a49f7-cb9c-4ddd-ae24-012d9dadffed)