MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit

MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit

Runs a tiny 0.8B reasoning-tuned chat model in the MLX format at 6-bit, converted from Jackrong's Qwen3.5 distilled model. For people who want a tiny local model on a Mac.

Made by Jackrong on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds a 6-bit MLX conversion of one of Jackrong's distilled Qwen3.5 0.8B models, a small text generation model. Hugging Face lists it at 752M parameters. The conversion used mlx-lm version 0.30.7.

Who it suits: It suits people who want a tiny language model for quick local tests, mostly on machines that run MLX.

Industry | Who uses it | How they use it | Example
Software | Developers | Test an MLX pipeline with a model that loads fast | Check that your generate call works end to end
Education | Students | Learn how local language models load and reply | Run a first prompt and read the output
AI hobbyists | Mac users | Try a compact model on a personal computer | Chat with a tiny model offline
Research | Tinkerers | Compare a 6-bit small model with larger ones | Note how short answers differ in quality

If you need a model for long, hard tasks, you can skip it, because it is small.

Where to use it:

• Python with the mlx-lm package
• Macs, since MLX is Apple's machine learning framework
• The download step works on Windows, macOS or Linux with Python

Use case | Who | What they get back
Quick local test | Developers | A reply to a short prompt printed in your terminal
Learning the tools | Students | A working load and generate example to build on
Offline chat | Mac users | Text answers with no cloud service

How to set it up: You need Python with pip and the mlx-lm package.

• Python and pip
• The mlx-lm package
• About 3 GB of disk, the size Hugging Face lists for the weights

1. Install the Hugging Face client with pip.
2. Download the repository with the hf command, or let the sample load it by name.
3. Install mlx-lm with pip.

pip install -U huggingface_hub
hf download Jackrong/MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit

pip install mlx-lm

How to use it: You run a short Python script. It loads the model by name, wraps your prompt in the chat template when the tokenizer has one, and prints the reply as it is made.

from mlx_lm import load, generate

model, tokenizer = load("Jackrong/MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

Tool | What it does
load | Loads the model and its tokenizer by name
apply_chat_template | Turns your message into the format the model expects
generate | Produces the reply, and prints it when verbose is on

Example: You keep the sample's prompt, the single word hello, and run the script. The prompt goes through the chat template, and the model prints a short greeting as its reply.

For newcomers and experts: MLX is a framework for running models on Apple hardware, and this is a tiny model stored in 6-bit form. The first small thing to try is the sample above, which needs only a few lines.

The card says little beyond the conversion. The model comes from a distilled Qwen3.5 0.8B model and was converted with mlx-lm, with the stored weights shown as U32 on the Hub. The trade-off is plain: a small, fast model with limited depth. Pick your own prompts to check quality.

Jackrong's Qwen3.5 0.8B distilled model
Convert with mlx-lm 0.30.7
6-bit MLX weights
Load and generate in Python

Good to know:

• Source model: the card names Jackrong's distilled Qwen3.5 0.8B model as the one it was converted from.
• Card detail: the card gives no benchmarks or training notes, so test it on your own tasks.
• License: Apache 2.0, as the License line says.
• Cost to run: no API key, account or paid service is named. It runs on your own computer.
• Data it reaches: the prompts you give it, and Hugging Face for the download.
• Not for: the card names no forbidden uses.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 752M
• Weights: about 3 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 253
• Likes: 4
• License: apache-2.0
• Languages: en
• Based on: Jackrong/Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled
• Tags: safetensors, qwen3_5, image-text-to-text, transformers, unsloth, text-generation, conversational, 6-bit

Links

• Model: https://huggingface.co/Jackrong/MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit
• Files: https://huggingface.co/Jackrong/MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Jackrong/MLX-Qwen3.5-0.8B-Claude-4.6-Opus-Reasoning-Distilled-6bit

Then follow the usage in its model card. Nothing in this kit runs by itself.