Qwen3.8-27B-oQ8e-mtp

Qwen3.8-27B-oQ8e-mtp

Runs Qwen3.8-27B as an 8-bit mixed-precision MLX checkpoint with its vision encoder and MTP head, for image and text chat on Apple hardware. For people who want near full quality locally.

Made by Jundot on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds an MLX checkpoint of Qwen/Qwen3.8-27B, quantized with the oQ method in oMLX. Besides the text model, the checkpoint carries the vision encoder and an MTP head for multi-token prediction. The average comes to 8.639 bits per weight.

Who it suits: It suits people who run large models on a Mac and want 8-bit quality, with image input and a speculative decoding head included.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local vision-language model on Apple hardware | Ask about a screenshot while offline
Research | ML researchers | Study mixed-precision quantization by layer sensitivity | Compare 8-bit oQ with uniform 8-bit
AI hobbyists | Local model users | Chat with a large model close to its original quality | Describe a photo and ask follow-up questions
Content | Creators | Get image and video understanding on a private machine | Summarize a short video on your own computer

If you have a smaller machine, you can skip it, since the checkpoint takes about 30 GB.

Where to use it:

• MLX-based tools on Apple silicon Macs
• oMLX, the tool the card names for this quantization
• Windows, macOS or Linux, where you download the files with Python

Use case | Who | What they get back
Image questions | Developers | Text answers about a picture or a video
Long context chat | Local model users | A model with a 262,144-token native context
Faster decoding | Developers | The MTP head, kept for speculative decoding

How to set it up: You need Python with pip, an MLX-based runner, and about 30 GB of disk.

• Python and pip
• An MLX-based runner, such as the oMLX tool named in the card
• About 30 GB of disk, from the card's storage table

1. Install the Hugging Face client with pip.
2. Download the repository with the hf command.
3. Load the folder in your MLX runner.
4. If you had an earlier copy, download again, because the card says the quantization was redone.

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ8e-mtp

How to use it: Once loaded in an MLX runner, you chat in text or add an image or video. The card shows no run commands, so follow your runner's own instructions.

Example: You load the checkpoint, attach a photo and ask what is in it. The model reads the picture with its vision encoder and answers in text. You can then ask follow-up questions in the same chat.

For newcomers and experts: Quantizing stores numbers with fewer bits. Here most weights use 8 bits, so quality stays close to the original while the file is smaller than the full BF16 model. The first small thing to try is the download command above.

The oQ8e method gives out bits by measured layer sensitivity. Each group is then rounded using an importance matrix built from 128 samples of 512 tokens, and its scale and bias are refit by weighted least squares. The base model mixes Gated DeltaNet and Gated Attention layers on the Qwen3.5 design.

Qwen3.8-27B original model
Measure sensitivity of each layer
Assign bits and round with an importance matrix
Refit scale and bias
MLX checkpoint with the MTP head

Good to know: Storage: the card gives 30.001 GB in total, and Hugging Face lists a larger figure in U32 units, so plan for the card's number.

Title: Storage by part in GB
Language backbone: 28.579 GB
Vision encoder: 0.921 GB
MTP head: 0.500 GB

Storage format | Share of weights | Bits per weight
Affine 8-bit, group 64 | 98.143% | 8.50
BF16 | 1.857% | 16.00

• Re-download: the card says the checkpoint was quantized again with an improved oQe method, so earlier copies should be replaced.
• License: Apache 2.0, as the License line says. The card says it follows the source model.
• Cost to run: no API key, account or paid service is named. You need hardware that fits about 30 GB.
• Data it reaches: the prompts, pictures and videos you give your runner, and Hugging Face for the download.
• Not for: the card names no forbidden uses.

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: mlx
• Parameters: 27.8B
• Weights: about 111 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 2,954
• Likes: 16
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: safetensors, qwen3_5, quantized, image-text-to-text, conversational, 8-bit

Links

• Model: https://huggingface.co/Jundot/Qwen3.8-27B-oQ8e-mtp
• Files: https://huggingface.co/Jundot/Qwen3.8-27B-oQ8e-mtp/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ8e-mtp

Then follow the usage in its model card. Nothing in this kit runs by itself.