Qwen3.8-27B-oQ6e-mtp

Qwen3.8-27B-oQ6e-mtp

A mixed-precision MLX copy of Qwen3.8-27B at about 6.9 bits per weight, with the vision encoder and multi-token prediction head included, for running locally on Apple silicon.

Made by Jundot on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is Qwen3.8-27B converted to MLX and compressed with oQ6e, a mixed-precision method from oMLX. The original is a dense 27B model that understands text, images and video. It follows the Qwen3.5 design and handles up to 262,144 tokens natively.

Its layers mix Gated DeltaNet and Gated Attention designs. The bundle holds the language model, the vision encoder and a head that predicts several tokens at once. The checkpoint averages 6.872 bits per weight overall.

Who it suits: People who run models locally with MLX and want a near-full-quality copy of a 27B multimodal model in about 22 GiB. Quantization researchers can also read its bit table.

Industry | Who uses it | How they use it | Example
Software | Local-model developers | Run a text and image model on their own machine | An offline assistant that reads images
Research | Quantization researchers | Study 6-bit and 8-bit allocation | Reading the card's bit distribution table
Hobbyists | MLX users | Chat with a large model locally | A private assistant on a Mac

You can skip it if you want a much smaller file, since this copy keeps high precision, or if you do not use MLX.

Where to use it: It is an MLX checkpoint for MLX-based tools on Apple silicon. The card names oMLX as the tool that made the quantization.

• MLX-based tools on Apple silicon
• oMLX, the quantization tool behind it
• Machines with room for roughly 22 GiB of weights

Use case | Who | What they get back
Local chat with images | Hobbyists | Replies to text and picture prompts
Long-context work | Developers | A model with a 262,144-token native context
Quantization comparison | Researchers | A 6-bit copy to compare with lower-bit ones

How to set it up: You need Python for the download tool, an MLX-based runner and about 22 GiB of space. The card gives only the download step and no run command.

• Python with pip
• An MLX-compatible runner
• About 22.2 GiB of disk space

1. Install the Hugging Face hub tools with pip.
2. Download the repository files with the hf command.
3. Open the checkpoint in your MLX runner.

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ6e-mtp

How to use it: Load the folder in an MLX-based tool and chat with it. The card gives no run command, so follow the steps of the tool you use. If you hold an older copy, the card says to download again.

Part | Storage format
Most weights | Affine 6-bit with group size 64
Some layers | Affine 8-bit with group size 64
Other components | BF16

Example: You open the checkpoint in an MLX runner and ask what a photographed whiteboard says. The model answers from its vision encoder and language backbone, entirely on your machine.

For newcomers and experts: If you are new, this is a ready-to-load model folder. Download it with the two commands above, open it in an MLX tool and begin with a short text question.

If you know the field, oQ6e spends bits according to measured layer sensitivity. It rounds each group with an importance matrix drawn from 128 samples of 512 tokens, then refits scale and bias. The language backbone averages 6.700 bits per weight.

Measure layer sensitivity
Assign 6-bit or 8-bit per group
Round with an importance matrix
Refit scale and bias
Save the MLX checkpoint

Good to know: About 88 percent of the weights are stored as 6-bit. The card gives the sizes below, for a total of about 23.865 GB.

Title: Storage by component, gigabytes
Language backbone: 22.526 GB
Vision encoder: 0.921 GB
MTP head: 0.418 GB

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the files you download, plus the prompts and images you give it locally
• Not for: tools outside MLX, since this is an MLX checkpoint

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: mlx
• Parameters: 27.8B
• Weights: about 111 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 2,536
• Likes: 21
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: safetensors, qwen3_5, quantized, image-text-to-text, conversational, 6-bit

Links

• Model: https://huggingface.co/Jundot/Qwen3.8-27B-oQ6e-mtp
• Files: https://huggingface.co/Jundot/Qwen3.8-27B-oQ6e-mtp/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ6e-mtp

Then follow the usage in its model card. Nothing in this kit runs by itself.