Qwen3-4B

Qwen3-4B

Generates text with a 4B Qwen3 model that switches between a thinking mode for hard reasoning and a plain mode for quick chat. Suits developers who want a small chat model.

Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3-4B is a 4B parameter text generation model from the Qwen3 series, built on Qwen3-4B-Base. One model covers two modes: a thinking mode for math, code and logic, and a non-thinking mode for everyday dialogue. It also works with external tools and supports 100+ languages and dialects.

Who it suits: Developers and teams who want a small open model for chat, reasoning or tool-using agents, hosted by themselves or run on a laptop with a local app.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a small chat and reasoning model in their own app | Serve it behind an OpenAI-compatible endpoint
Localization | Translators | Handle instructions and translation across many languages | Ask it to translate a short note
Agents | Agent builders | Let the model call external tools | Use thinking mode for a multi-step task

If you need a model with far more parameters, you can skip this one.

Where to use it:

• Python with the transformers library
• SGLang or vLLM, which serve an OpenAI-compatible endpoint
• Local apps such as Ollama and LMStudio

Use case | Who | What they get back
Reasoning in thinking mode | Students and developers | Step-by-step reasoning on math, code and logic
Quick chat in non-thinking mode | App builders | Fast general-purpose replies
Creative writing and role-play | Writers | Natural multi-turn conversation

How to set it up:

• Python with pip
• A recent transformers release, since versions below 4.51.0 fail with a KeyError on qwen3
• Roughly 8 GB for the BF16 weights, or about 2 GB at 4-bit by Hugging Face's estimate

1. Install the Hugging Face hub package.
2. Download the model files.
3. Load the model with transformers, or serve it with SGLang or vLLM.

pip install -U huggingface_hub
hf download Qwen/Qwen3-4B

To serve it as an endpoint, the card gives these commands:

python -m sglang.launch_server --model-path Qwen/Qwen3-4B --reasoning-parser qwen3

vllm serve Qwen/Qwen3-4B --enable-reasoning --reasoning-parser deepseek_r1

How to use it: Load the tokenizer and model, build a chat message list, and apply the chat template. The enable_thinking flag switches between thinking and non-thinking modes, and it defaults to True.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen3-4B"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

Tool | What it does
AutoTokenizer | Loads the tokenizer and applies the chat template
AutoModelForCausalLM | Loads the model and generates the reply
enable_thinking | Switches thinking mode on or off

Example: You send the message asking for a short introduction to large language models. In thinking mode the reply holds the reasoning first and the answer after it, and you can parse the two parts apart.

For newcomers and experts: If you are new, load the model with the snippet above and send one short question. Leave thinking on for hard problems and turn it off when you only want a quick answer.

The model has 36 layers and grouped query attention, 32 heads for queries and 8 for keys and values. Context is 32,768 tokens natively and 131,072 with YaRN. If you see endless repetition, the card advises its sampling settings and a presence_penalty of 1.5.

Download the files or load them with transformers
Build messages and apply the chat template
Choose thinking or non-thinking mode
Generate and read the reply

Good to know:

• Context: 32,768 tokens natively, 131,072 with YaRN
• Use the latest transformers release to avoid the qwen3 error
• License: Apache 2.0
• Cost to run: none named, you supply the hardware
• Data it reaches: your prompts and the files downloaded from Hugging Face
• Not for: transformers versions below 4.51.0

Title: Context length in tokens
Native: 32768 tokens
With YaRN: 131072 tokens

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 4B
• Weights: about 8 GB in BF16; about 2 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 3,186,531
• Likes: 798
• License: apache-2.0
• Based on: Qwen/Qwen3-4B-Base
• Quantized versions: 382
• Fine-tunes: 1,262
• Tags: transformers, safetensors, qwen3, text-generation, conversational

Links

• Model: https://huggingface.co/Qwen/Qwen3-4B
• Files: https://huggingface.co/Qwen/Qwen3-4B/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Qwen/Qwen3-4B

Then follow the usage in its model card. Nothing in this kit runs by itself.