Qwen3-1.7B-GGUF

Qwen3-1.7B-GGUF

Gives you the small Qwen3 1.7B chat model as a q8_0 GGUF file for llama.cpp, with a switch between thinking and non-thinking replies. For people who want a light local assistant.

Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds Qwen3-1.7B in GGUF format with q8_0 quantization, for llama.cpp. Qwen3 is the latest Qwen generation, and this is its small 1.7B member.

Its main feature is a choice between two modes in one model. Thinking mode works through logic, math and code step by step, and non-thinking mode answers general chat quickly. The card also lists support for more than 100 languages and dialects, and tool use.

Who it suits: It suits people who want a small assistant on their own machine, with the option to turn deeper reasoning on or off per message.

Industry | Who uses it | How they use it | Example
Local AI | Enthusiasts | Chat with a small model offline | Ask quick questions
Software | Developers | Run a light model in a local tool | Draft a short script
Education | Students | Practice math and logic with visible reasoning | Work a counting puzzle
Localization | Translators | Try translation across many languages | Translate a short note

If you need the strongest answers, you can skip it, since larger Qwen3 models exist.

Where to use it: The file runs in llama.cpp on Windows, macOS or Linux. The card also shows ollama as a way to run it.

• llama-cli from llama.cpp
• ollama, with one command
• A laptop CPU or a GPU with layers offloaded

Use case | Who | What they get back
Quick chat | Everyday users | Fast replies in non-thinking mode
Reasoning tasks | Students | A visible thinking trace, then an answer
Multilingual help | Translators | Replies and translations in many languages

How to set it up: You need Python with pip to fetch the file, and a working llama.cpp. The card says to clone llama.cpp and install it by its guide.

• Python with pip
• llama.cpp installed
• About 1.8 GB of disk space for the q8_0 file, estimated from the model size

1. Install the Hugging Face hub package.
2. Download the GGUF file.
3. Run it with llama-cli from the llama.cpp folder.

pip install -U huggingface_hub
hf download Qwen/Qwen3-1.7B-GGUF

How to use it: You start llama-cli with the model and the sampling settings the card recommends, then chat. Add /think or /no_think to a message to switch modes for that turn.

./llama-cli -hf Qwen/Qwen3-1.7B-GGUF:Q8_0 --jinja --color -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 1.5 -c 40960 -n 32768 --no-context-shift

Switch | What it does
/think | Turns on thinking mode for that message
/no_think | Turns it off for that message

Example: You ask a counting puzzle and add /think to the end. The model writes a thinking trace first and then gives the answer. For a simple question you add /no_think and get a short reply.

For newcomers and experts: If you are new, run the command and ask a simple question, then ask a puzzle with /think and compare the two replies.

If you know the field, the model has 28 layers and grouped-query attention with 16 query heads and 8 key-value heads. Context is 32,768 tokens. The card advises against greedy decoding in thinking mode, since it can cause endless repetition.

Start llama-cli with the GGUF file
Send a message
Add /think or /no_think
Read the reply

Good to know: The card suggests a presence penalty of 1.5 for quantized models to cut repeated text. Higher values can mix languages and slightly lower quality.

• License: apache-2.0
• Cost to run: none named
• Data it reaches: files you download, and the messages you type
• Not for: greedy decoding in thinking mode, which the card warns against

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 1.7B
• Weights: about 860 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 177,416
• Likes: 98
• License: apache-2.0
• Based on: Qwen/Qwen3-1.7B
• Quantized versions: 2
• Tags: gguf, text-generation, conversational

Links

• Model: https://huggingface.co/Qwen/Qwen3-1.7B-GGUF
• Files: https://huggingface.co/Qwen/Qwen3-1.7B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Qwen/Qwen3-1.7B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.