Qwen3-30B-A3B-GGUF

Qwen3-30B-A3B-GGUF

A 30.5B mixture-of-experts chat model in GGUF format with only 3.3B parameters active per token, runnable in llama.cpp or ollama. It has thinking and plain modes and suits developers with a roomy local machine.

Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3-30B-A3B-GGUF is a mixture-of-experts chat model from Qwen in GGUF format, so it runs locally in llama.cpp or ollama. It holds 30.5 billion parameters, but only 3.3 billion are active for each token. One model offers a thinking mode and a plain dialogue mode.

Who it suits: People with a machine that has room for a larger local model, and developers who want strong reasoning, code and multilingual answers without a hosted service. Agent builders can use it for tool calls in both modes.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a larger local model for code and reasoning | Ask for a function and a walk-through
Developer tools | Agent builders | Let the model call external tools | Wire it into a local agent loop
Localization | Multilingual teams | Translate and follow instructions in many languages | Draft translations for review
Research | Researchers and hobbyists | Study a mixture-of-experts model locally | Test long context with YaRN

Anyone with a small laptop can skip it, because even the 4-bit weights are an estimated 15.3 GB.

Where to use it:

• llama.cpp on Windows, macOS or Linux, through its llama-cli tool
• ollama, which runs the model with one command

Use case | Who | What they get back
Local reasoning assistant | Developers and students | A thinking section, then the final answer
Code help | Developers | Generated and explained code
Long text work | Teams with big inputs | Context up to 131,072 tokens once YaRN is on
Multilingual replies | Global teams | Answers and translation in 100+ languages and dialects

How to set it up:

• Python with pip, if you want the files on disk
• llama.cpp installed by following its official guide, or ollama
• Memory for the file you choose, since a 4-bit copy is an estimated 15.3 GB

1. Install the Hugging Face hub package and download the files.
2. Or skip the download and run the model straight from llama.cpp or ollama with the commands below.

pip install -U huggingface_hub
hf download Qwen/Qwen3-30B-A3B-GGUF

How to use it: Run it from the llama.cpp folder, or with one ollama command, and chat in the terminal. Add /think or /no_think to a message to switch the thinking mode for that turn. The model follows the latest instruction in a multi-turn chat.

./llama-cli -hf Qwen/Qwen3-30B-A3B:Q8_0 --jinja --color -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 1.5 -c 40960 -n 32768 --no-context-shift

ollama run hf.co/Qwen/Qwen3-30B-A3B-GGUF:Q8_0

Tool | What it does
llama-cli | Runs the GGUF model in the terminal with the settings you pass
ollama run | Pulls the model and starts a chat with one command
/think | Turns thinking mode on for the current and later turns
/no_think | Turns thinking mode off for quick replies

To handle inputs past 32,768 tokens in llama.cpp, add the YaRN options. They keep the scaling constant for every input, which can slightly hurt shorter texts, so switch them on only when you need long context.

./llama-cli ... -c 131072 --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 32768

Example: In a chat you type "How many 'r's are in 'strawberries'? /think". The model works through the letters in a thinking section, then states that the word has 3 instances of the letter r.

For newcomers and experts: If you are new, run the ollama command, ask one question with /no_think for a fast reply, and ask it again with /think to see the reasoning. Expect a large download.

Experts get a causal language model with 48 layers, 32 query heads and 4 key-value heads (GQA). It has 128 experts with 8 active per token, and about 29.9B non-embedding parameters. Native context is 32,768 tokens, and 131,072 with YaRN.

You start llama-cli or ollama
The GGUF model loads locally
Eight of 128 experts handle each token
You get a reply, thinking first if asked

Good to know:

• Quantization: five levels are published, from q4_K_M up to q8_0
• Active size: 3.3 billion of 30.5 billion parameters work on each token
• Context: 32,768 tokens natively, 131,072 with YaRN
• License: Apache 2.0
• Cost to run: none named, no API key or account, only hardware with enough memory
• Data it reaches: it downloads model files from Hugging Face, then answers on your own machine
• Not for: the source names no forbidden uses, and YaRN may slightly hurt performance on shorter texts

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 30.5B
• Weights: about 15.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 131,118
• Likes: 93
• License: apache-2.0
• Based on: Qwen/Qwen3-30B-A3B
• Tags: gguf, text-generation, conversational

Links

• Model: https://huggingface.co/Qwen/Qwen3-30B-A3B-GGUF
• Files: https://huggingface.co/Qwen/Qwen3-30B-A3B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Qwen/Qwen3-30B-A3B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.