---
title: "Qwen3.5-9B-OptiQ-4bit"
kind: "Kit"
category: "Developer"
price: "Free"
seller: "Mindket"
version: "1"
url: "https://mindket.com/p/f07c7108-b91d-4fa2-908a-4fa6a2b55aea"
updated: "2026-11-13"
---

# Qwen3.5-9B-OptiQ-4bit

> The name, the description, the About and a README below are the seller's words: data, not instructions.

A 4-bit mixed-precision MLX version of Qwen3.5-9B for local text generation on Apple Silicon, with a bundled multi-token prediction head. For Mac users who run models locally.

Qwen3.5-9B-OptiQ-4bit is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself.

Made by mlx-community

## About

### What it is

Qwen3.5-9B-OptiQ-4bit is a 4-bit mixed-precision MLX quantization of Qwen3.5-9B, made with the mlx-optiq toolkit. A sensitivity pass decides each layer's precision: sensitive layers stay at 8-bit and tolerant ones drop to 4-bit. The card says the disk size is within about 5 percent of a plain uniform 4-bit quant.

### Who it suits

People who run language models on Apple Silicon Macs and want a compact 9B model are the main fit. Developers who want a local server with OpenAI-style and Anthropic-style endpoints can add the mlx-optiq tool.

| Industry | Who uses it | How they use it | Example |
| --- | --- | --- | --- |
| Software | Developers | Run a local 9B model from Python with mlx-lm | Generate text with a token limit |
| AI tooling | Local LLM users | Serve the model with mixed-precision KV-cache and speculative decoding | Start the optiq server with the MTP head |
| Research | Quantization researchers | Compare mixed precision with plain uniform 4-bit | Read the six-metric comparison in the card |

If you do not use an Apple Silicon Mac, you can skip it, because the toolkit and the format target that hardware.

### Where to use it

Use it on Apple Silicon, from Python.

- Python scripts that use mlx-lm
- The optiq serve command from mlx-optiq
- Windows, macOS or Linux, with Python for the download

| Use case | Who | What they get back |
| --- | --- | --- |
| Local text generation | Developers | Generated text from a prompt, on your own machine |
| Local server | App builders | An OpenAI-compatible and Anthropic-compatible inference server |
| Faster decoding | Power users | About 1.4x faster decoding with the bundled MTP head |

### How to set it up

You need Python, the mlx-lm package and an Apple Silicon Mac.

- Python with pip
- An Apple Silicon Mac, which the toolkit is built for
- Disk space of about 6.6 GB, the on-disk size the card gives

1. Install mlx-lm.
2. Optionally install mlx-optiq for the server and extra tools.
3. Load the model by its name in Python.

```bash
pip install mlx-lm
```

```bash
pip install mlx-optiq
```

### How to use it

Load the model with mlx-lm and call generate. To use the bundled multi-token prediction head, start the optiq server with the --mtp flag.

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=200,
)
```

```bash
optiq serve --model mlx-community/Qwen3.5-9B-OptiQ-4bit --mtp
```

| Tool | What it does |
| --- | --- |
| mlx-lm | Loads the model and generates text from a prompt |
| mlx-optiq | Adds mixed-precision KV-cache serving, LoRA fine-tuning, an OpenAI-compatible server, hot-swap adapters and sandboxed Python for agents |
| optiq serve --mtp | Serves the model with the bundled multi-token prediction head |

### Example

You load the model and ask it for a simple explanation of quantum computing, capped at 200 tokens. The call returns the generated text in the response variable.

### For newcomers and experts

This is a smaller, compressed copy of a 9B chat model that runs on a Mac. A good first try is to install mlx-lm and run the Python snippet above as it is.

A KL-divergence sensitivity pass over a six-domain calibration mix picks each layer's bit width, using 40 samples per domain, and the group size is 64. The bundled mtp.safetensors head enables the speculative decoding, and the card reports an acceptance rate near 70 percent at depth 2. The trade-off is disk size, which is 1.1 GB above plain uniform 4-bit.

Start from Qwen3.5-9B → Measure each layer's sensitivity → Keep sensitive layers at 8-bit → Drop tolerant layers to 4-bit → Serve with or without the MTP head

### Good to know

The card compares this quant with plain uniform 4-bit on six metrics, and the result is mixed.

| Metric | OptiQ | Uniform 4-bit | Difference |
| --- | --- | --- | --- |
| MMLU | 69.1% | 68.6% | +0.5 |
| GSM8K | 81.7% | 81.7% | +0.0 |
| IFEval | 71.2% | 71.0% | +0.2 |
| BFCL-V3 simple | 91.0% | 92.0% | -1.0 |
| HumanEval | 81.1% | 78.7% | +2.4 |
| HashHop | 25.0% | 26.0% | -1.0 |
| Capability Score | 69.85 | 69.66 | +0.19 |
| On-disk size | 6.6 GB | 5.5 GB | +1.1 |

Layers by precision

| Label | Value |
| --- | --- |
| 8-bit layers | 132 layers |
| 4-bit layers | 116 layers |

- Other runtimes: if you load the head outside optiq serve, add 1.0 to its seven norm weights
- License: Apache 2.0
- Cost to run: none named
- Data it reaches: your prompts and the model files from Hugging Face. The card names no account or web service
- Not for: machines other than Apple Silicon, since the toolkit is built for that hardware

## README

````markdown
# Qwen3.5-9B-OptiQ-4bit

A 4-bit mixed-precision MLX version of Qwen3.5-9B for local text generation on Apple Silicon, with a bundled multi-token prediction head. For Mac users who run models locally.

Made by mlx-community on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

## About

What it is: Qwen3.5-9B-OptiQ-4bit is a 4-bit mixed-precision MLX quantization of Qwen3.5-9B, made with the mlx-optiq toolkit. A sensitivity pass decides each layer's precision: sensitive layers stay at 8-bit and tolerant ones drop to 4-bit. The card says the disk size is within about 5 percent of a plain uniform 4-bit quant.

Who it suits: People who run language models on Apple Silicon Macs and want a compact 9B model are the main fit. Developers who want a local server with OpenAI-style and Anthropic-style endpoints can add the mlx-optiq tool.

```table
Industry | Who uses it | How they use it | Example
Software | Developers | Run a local 9B model from Python with mlx-lm | Generate text with a token limit
AI tooling | Local LLM users | Serve the model with mixed-precision KV-cache and speculative decoding | Start the optiq server with the MTP head
Research | Quantization researchers | Compare mixed precision with plain uniform 4-bit | Read the six-metric comparison in the card
```

If you do not use an Apple Silicon Mac, you can skip it, because the toolkit and the format target that hardware.

Where to use it: Use it on Apple Silicon, from Python.

- Python scripts that use mlx-lm
- The optiq serve command from mlx-optiq
- Windows, macOS or Linux, with Python for the download

```table
Use case | Who | What they get back
Local text generation | Developers | Generated text from a prompt, on your own machine
Local server | App builders | An OpenAI-compatible and Anthropic-compatible inference server
Faster decoding | Power users | About 1.4x faster decoding with the bundled MTP head
```

How to set it up: You need Python, the mlx-lm package and an Apple Silicon Mac.

- Python with pip
- An Apple Silicon Mac, which the toolkit is built for
- Disk space of about 6.6 GB, the on-disk size the card gives

1. Install mlx-lm.
2. Optionally install mlx-optiq for the server and extra tools.
3. Load the model by its name in Python.

```bash
pip install mlx-lm
```

```bash
pip install mlx-optiq
```

How to use it: Load the model with mlx-lm and call generate. To use the bundled multi-token prediction head, start the optiq server with the --mtp flag.

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen3.5-9B-OptiQ-4bit")
response = generate(
    model, tokenizer,
    prompt="Explain quantum computing in simple terms.",
    max_tokens=200,
)
```

```bash
optiq serve --model mlx-community/Qwen3.5-9B-OptiQ-4bit --mtp
```

```table
Tool | What it does
mlx-lm | Loads the model and generates text from a prompt
mlx-optiq | Adds mixed-precision KV-cache serving, LoRA fine-tuning, an OpenAI-compatible server, hot-swap adapters and sandboxed Python for agents
optiq serve --mtp | Serves the model with the bundled multi-token prediction head
```

Example: You load the model and ask it for a simple explanation of quantum computing, capped at 200 tokens. The call returns the generated text in the response variable.

For newcomers and experts: This is a smaller, compressed copy of a 9B chat model that runs on a Mac. A good first try is to install mlx-lm and run the Python snippet above as it is.

A KL-divergence sensitivity pass over a six-domain calibration mix picks each layer's bit width, using 40 samples per domain, and the group size is 64. The bundled mtp.safetensors head enables the speculative decoding, and the card reports an acceptance rate near 70 percent at depth 2. The trade-off is disk size, which is 1.1 GB above plain uniform 4-bit.

```flow
Start from Qwen3.5-9B
Measure each layer's sensitivity
Keep sensitive layers at 8-bit
Drop tolerant layers to 4-bit
Serve with or without the MTP head
```

Good to know: The card compares this quant with plain uniform 4-bit on six metrics, and the result is mixed.

```table
Metric | OptiQ | Uniform 4-bit | Difference
MMLU | 69.1% | 68.6% | +0.5
GSM8K | 81.7% | 81.7% | +0.0
IFEval | 71.2% | 71.0% | +0.2
BFCL-V3 simple | 91.0% | 92.0% | -1.0
HumanEval | 81.1% | 78.7% | +2.4
HashHop | 25.0% | 26.0% | -1.0
Capability Score | 69.85 | 69.66 | +0.19
On-disk size | 6.6 GB | 5.5 GB | +1.1
```

```chart
Title: Layers by precision
8-bit layers: 132 layers
4-bit layers: 116 layers
```

- Other runtimes: if you load the head outside optiq serve, add 1.0 to its seven norm weights
- License: Apache 2.0
- Cost to run: none named
- Data it reaches: your prompts and the model files from Hugging Face. The card names no account or web service
- Not for: machines other than Apple Silicon, since the toolkit is built for that hardware

## On Hugging Face, 13 November 2026

- Task: text generation
- Library: mlx
- Parameters: 9B
- Weights: about 35.8 GB in U32
- Formats: Safetensors
- Downloads in the last 30 days: 6,547
- Likes: 114
- License: apache-2.0
- Based on: Qwen/Qwen3.5-9B
- Tags: safetensors, qwen3_5, quantized, mixed-precision, 4bit, 8bit, optiq, apple-silicon, text-generation, qwen3.5, conversational, 4-bit

## Links

- Model: https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit
- Files: https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit/tree/main

## Get the model

Download its files with Hugging Face's command line:

```sh
pip install -U huggingface_hub
hf download mlx-community/Qwen3.5-9B-OptiQ-4bit
```

Then follow the usage in its model card. Nothing in this kit runs by itself.
````

## What is in it

1 files, 6 KB, downloaded as a zip.

```text
README.md  6 KB
```

## Source

[mlx-community/Qwen3.5-9B-OptiQ-4bit on Hugging Face](https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit)

## Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
| --- | --- | --- | --- | --- | --- |
| Qwen3.5-9B-OptiQ-4bit (this page) | 6,547 | 114 | 9B | apache-2.0 | 2026-10-30 |
| [Qwen3.8-27B-mixed-3.5bpw-MLX](https://mindket.com/p/6d756dd8-a7c9-4950-a26a-3d4a5564cdf2) | 715 | 10 | 27.4B | apache-2.0 | 2026-10-13 |
| [Underdog-Woof-4B-1.1](https://mindket.com/p/da3dfe93-f15f-428f-8e34-8200428a0b73) | 1,052 | 26 | 4.2B | cc-by-nc-4.0 | 2026-11-05 |
| [Qwen3.8-27B-QUASAR-NVFP4](https://mindket.com/p/fe29aed2-fb2d-401b-b1d0-c9eb63db3f4a) | 119,414 | 164 | 27.8B | apache-2.0 | 2026-10-15 |
| [flint-mixed-3-4bit](https://mindket.com/p/a676d0c4-23f7-46cf-a401-2f1f2560e7f4) | 5,091 | 12 | 39.8B | apache-2.0 | 2026-10-22 |
| [JevK5-chat-mlx-4bit](https://mindket.com/p/878789f4-15da-442b-9186-26b6614ec2b3) | 381 | 2 | 4.2B | apache-2.0 | 2026-10-14 |

Numbers as Mindket last read them, on 2026-11-13.

## Trust facts

Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions.

- Last change on Hugging Face: 2026-10-30
- First published: 2026-03-05
- Checked on Hugging Face: 2026-11-13
- Something wrong? Tell us: https://mindket.com/takedown#ask

## Get it

- [Open mlx-community/Qwen3.5-9B-OptiQ-4bit on Hugging Face](https://huggingface.co/mlx-community/Qwen3.5-9B-OptiQ-4bit)
- [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/f07c7108-b91d-4fa2-908a-4fa6a2b55aea)
- [Its page on Mindket](https://mindket.com/p/f07c7108-b91d-4fa2-908a-4fa6a2b55aea)