Qwen3.8-27B-oQ4e-mtp

Qwen3.8-27B-oQ4e-mtp

Runs the 27B Qwen3.8 vision-language model on MLX in about 17.5 GB, reading text, images and video with a 262,144-token context. A mixed-precision quantization by Jundot for people with limited memory.

Made by Jundot on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3.8-27B-oQ4e-mtp is a compressed MLX copy of the 27B Qwen3.8 vision-language model, made by Jundot with oMLX. Bits go to each layer by how sensitive it is, so the whole checkpoint averages 5.044 bits per weight. Three parts ship together: the text model, a vision encoder for images and video, and an MTP (multi-token prediction) head.

Who it suits: People who run large multimodal models on their own machine through MLX and want text, image and video understanding at a size that fits in less memory. The files carry the Apache 2.0 license.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a multimodal model locally in MLX apps | Asking questions about a screenshot
Research | Quantization researchers | Study sensitivity-based mixed precision | Reading the bit distribution table
Productivity | Power users | Keep a large vision-language model on a personal machine | Describing the contents of a video clip

Anyone who needs a runtime other than MLX can skip it, since this checkpoint is built for MLX.

Where to use it:

• MLX, the library Hugging Face lists, which is built for Apple Silicon Macs
• MLX-based tools such as oMLX, the project that made this quantization
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Image questions | Developers | Text answers about an image you supply
Video understanding | Researchers | Text that describes what happens in a clip
Long inputs | Power users | A native context of 262,144 tokens

How to set it up:

• Python and the huggingface_hub package, to download the files
• An MLX setup that can load a vision-language checkpoint
• Room for the files, about 17.517 GB in total

1. Install Python and your MLX tools if you do not have them yet.
2. Install the huggingface_hub package with the first command below.
3. Download the model files with the second command.
4. If you hold an older copy, download this one again, because the card says it was re-quantized on 2026-10-29.

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ4e-mtp

How to use it: Once the files are on your computer, you load the checkpoint in your MLX tool and send it text, images or video. The card text shows no run command, so follow your tool's steps for loading a vision-language model.

Example: You load the checkpoint, attach a photo and ask what it shows. The model reads the image with its vision encoder and answers in text.

For newcomers and experts: Quantization stores a model's numbers with fewer bits so it fits in less memory. A good first step is to download the files and load them in your MLX tool.

The oQ4e method sets bits by measured layer sensitivity. It then rounds each group using an importance matrix of 128 samples of 512 tokens, and refits scale and bias by weighted least squares. About 70 percent of the weights sit at 4 bits, and the rest use 5, 6 or 8 bits, or BF16.

Measure each layer's sensitivity
Assign 4, 5, 6 or 8 bits
Round with an importance matrix
Refit scale and bias
Save the MLX checkpoint

Good to know:

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the download fetches files from Hugging Face, and the source names nothing else
• The card says it was re-quantized with the improved oQe method in oMLX 0.7.3
• Hugging Face lists the weights as about 111 GB in U32, while the card puts the whole checkpoint at 17.517 GB

Title: Share of weights by storage format, in percent
Affine 4-bit: 69.807%
Affine 5-bit: 23.680%
Affine 6-bit: 4.615%
BF16: 1.857%
Affine 8-bit: 0.041%

Title: Storage by component in GB
Language backbone: 16.272 GB
Vision encoder: 0.921 GB
MTP head: 0.323 GB

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: mlx
• Parameters: 27.8B
• Weights: about 111 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 37,954
• Likes: 86
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: safetensors, qwen3_5, quantized, image-text-to-text, conversational, 4-bit

Links

• Model: https://huggingface.co/Jundot/Qwen3.8-27B-oQ4e-mtp
• Files: https://huggingface.co/Jundot/Qwen3.8-27B-oQ4e-mtp/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Jundot/Qwen3.8-27B-oQ4e-mtp

Then follow the usage in its model card. Nothing in this kit runs by itself.