Qwen3-8B-MLX-bf16

Qwen3-8B-MLX-bf16

Runs the Qwen3 8B language model through the mlx library for chat, reasoning, translation and agent work. Suits developers who want a local text model with a switchable thinking mode.

Made by Qwen on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3-8B-MLX-bf16 is the 8.2B parameter Qwen3 language model, stored in BF16 for the mlx library. It writes text from prompts and handles reasoning, instructions, translation and tool use. One model can run in a thinking mode for logic, math and code, or in a non-thinking mode for everyday dialogue.

Who it suits: Developers who want a mid-sized language model they can call from Python through mlx_lm are the main fit. Writers, translators and agent builders can use it too, since the card points to creative writing, over 100 languages and tool integration.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a chat model from Python through mlx_lm | Generate a reply to a prompt with a token limit
Language services | Translators | Follow instructions and translate across many languages | Translate a short paragraph between two languages
Creative work | Writers | Draft stories, role-play and multi-turn dialogue | Write a scene in the voice of a character
Agents | Agent builders | Connect the model to external tools in either mode | Let an agent pick a tool for a task

If your machine cannot hold about 16.4 GB of weights, you can skip this BF16 copy.

Where to use it: Use it from Python on Windows, macOS or Linux.

• Python scripts that use the mlx_lm package
• Python scripts that use the transformers library
• APIs built with SGLang or vLLM, where the thinking switch also exists

Use case | Who | What they get back
Chat and Q and A | Developers | A written reply, with an optional think block before the answer
Translation | Translators | Text moved between many languages and dialects
Tool-using agents | Agent builders | Model output that can call external tools in thinking or non-thinking mode

How to set it up: You need Python and recent versions of two packages.

• Python on Windows, macOS or Linux
• transformers 4.52.4 or newer and mlx_lm 0.25.2 or newer
• Room for the weights, about 16.4 GB in BF16 or about 4.1 GB at 4-bit as an estimate

1. Install or upgrade transformers and mlx_lm together.
2. Optionally download the model files first with the Hugging Face hub package.
3. Run the Python snippet from the next section.

pip install --upgrade transformers mlx_lm

pip install -U huggingface_hub
hf download Qwen/Qwen3-8B-MLX-bf16

How to use it: Load the model and tokenizer, wrap your message in the chat template, and call generate. The snippet prints the reply.

from mlx_lm import load, generate

model, tokenizer = load("Qwen/Qwen3-8B-MLX-bf16")
prompt = "Hello, please introduce yourself and tell me what you can do."

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages,
        add_generation_prompt=True
    )

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    verbose=True,
    max_tokens=1024
)

print(response)

Tool | What it does
load | Loads the model and its tokenizer from the model name
generate | Produces a reply from a prompt, up to a token limit
apply_chat_template | Turns your messages into a chat prompt and can switch thinking on or off

Thinking is on by default. The card shows how to set it explicitly, and the reply then holds a think block followed by the final answer.

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True  # True is the default value for enable_thinking
)

For thinking mode, the card advises a temperature of 0.6, a top-p of 0.95, a top-k of 20 and a min-p of 0, and warns against greedy decoding.

Example: You ask the model to introduce itself and say what it can do. The script prints a reply of up to 1024 tokens. With thinking on, the reply starts with a think block and then gives the answer.

For newcomers and experts: This is a chat model you run from a short Python script. A good first try is to install the two packages, run the snippet above as it is, and read the reply it prints.

Under the hood it is a dense causal language model with 36 layers and grouped-query attention, using 32 query heads and 8 key-value heads. Its context is 32,768 tokens natively and 131,072 with YaRN. Thinking mode spends extra output on reasoning before the answer.

Install transformers and mlx_lm
Load the model and tokenizer
Wrap your message in the chat template
Generate up to the token limit
Read the reply, think block first

Good to know: Older transformers versions can fail with a KeyError for qwen3. One quantized copy and no fine-tuned copies by other people are listed.

Title: Context length
Native: 32,768 tokens
With YaRN: 131,072 tokens

• License: Apache 2.0
• Cost to run: none named. The weights take about 16.4 GB in BF16 or about 4.1 GB at 4-bit, as an estimate
• Data it reaches: your prompts and the model files from Hugging Face. The source names no account or web service
• Not for: greedy decoding in thinking mode, which the card says can cause weaker output and endless repetition

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 8.2B
• Weights: about 16.4 GB in BF16; about 4.1 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 697
• Likes: 19
• License: apache-2.0
• Quantized versions: 1
• Tags: safetensors, qwen3, text-generation, conversational

Links

• Model: https://huggingface.co/Qwen/Qwen3-8B-MLX-bf16
• Files: https://huggingface.co/Qwen/Qwen3-8B-MLX-bf16/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Qwen/Qwen3-8B-MLX-bf16

Then follow the usage in its model card. Nothing in this kit runs by itself.