Qwen3.8-27B-mixed-3.5bpw-MLX

Qwen3.8-27B-mixed-3.5bpw-MLX

A 27B vision-language model compressed to 3.5 bits per weight for Apple Silicon Macs with 24 GB of memory, served locally through Rapid-MLX. Suits Mac users and developers who want a local model and agent backend.

Made by rapid-mlx on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is an MLX build of Qwen3.8-27B for Apple Silicon Macs, squeezed to an average of 3.5 bits per weight. It takes text and images and answers in text. The card says it fits on a 24 GB Mac while keeping reasoning, knowledge and code close to a standard 4-bit build.

Who it suits: Mac owners who want a 27B vision-language model running on their own machine, and developers who want a local model behind coding agents. You need an Apple Silicon Mac with at least 24 GB of unified memory.

Industry | Who uses it | How they use it | Example
Software | Developers on Macs | Run a large local model for coding help | Connect it to a coding agent
AI applications | Agent builders | Serve an OpenAI-compatible endpoint locally | Point an app at the local server
Research | Quantization researchers | Study mixed-precision allocation | Compare the 3.5 bpw build with uniform 3-bit
Everyday use | Privacy-minded users | Chat with a model that never leaves the computer | Ask questions about an image

Anyone without an Apple Silicon Mac can skip it, since the build targets MLX and Apple unified memory.

Where to use it:

• Apple Silicon Macs, with 24 GB unified memory as the minimum and 32 GB giving more room for context
• Rapid-MLX, which serves the model with an OpenAI-compatible interface
• OpenCode, Claude Code and Codex, which the card says can connect through the agents setup command

Use case | Who | What they get back
Local text chat | Mac users | Replies generated on your own machine
Image questions | Mac users | Text answers about a picture, when the vision option is on
Agent backend | Developers | A local model for coding agents
Faster decoding | Advanced users | Optional speculative decoding with the bundled MTP drafter

How to set it up:

• An Apple Silicon Mac with at least 24 GB of unified memory
• Python with pip
• Rapid-MLX, which runs the model, plus about 13 GB of free disk space

1. Install the Hugging Face hub package and download the files, if you want them on disk.
2. Install Rapid-MLX with pip.
3. Start the server with one of the serve commands below.

pip install -U huggingface_hub
hf download rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX

pip install rapid-mlx

How to use it: Start the server in text mode, or add the vision flag to include images. Then send chat requests to the local OpenAI-compatible port, which is 8000 in these commands.

rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000            # text
rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000 --mllm     # + vision

The short form below uses the model's built-in name.

rapid-mlx serve qwen3.8-27b-mixed-3.5bpw

The repository also holds a paired Multi-Token Prediction drafter. You can switch on lossless speculative decoding with this command. It is off by default because the speed gain differs between Apple chips.

rapid-mlx serve qwen3.8-27b-mixed-3.5bpw \
  --speculative-config '{"method":"mtp","model":"rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX","num_speculative_tokens":3}'

Tool | What it does
rapid-mlx serve | Starts the local server for the model
--mllm | Adds vision, so the server accepts images
--speculative-config | Switches on the optional MTP speculative decoding
rapid-mlx agents <name> --setup | Connects the server to a coding agent such as OpenCode, Claude Code or Codex

Example: After the server is running, you send a chat request asking the model to say hello. It answers with a short greeting, and the same request format works from any OpenAI-compatible client.

For newcomers and experts: If you are new, check your Mac has 24 GB of memory, start the text-only server, and send a simple greeting before you try images or agents. The first start is the longest because the weights are large.

Experts get an activation-aware mixed-precision allocation. A calibration run measures how much each language layer moves the outputs. Layers that matter most get more bits, from 2 to 4, and the average target is 3.5 bits per weight. The vision tower stays at 4-bit. Group size is 64.

Title: MMLU-Pro score by build
4-bit build: 60.8 points
This 3.5 bpw build: 61.0 points
Uniform 3-bit build: 54.2 points

Title: Reported scores from the card's own tests
Benchmark | 4-bit | This build | Uniform 3-bit
MMLU-Pro | 60.8 | 61.0 | 54.2
GSM8K | 95.3 | 94.7 | 92.7
HumanEval+ | 77.5 | 80.0 | none
MMBench-EN | 90.3 | 86.7 | 87.3

Install Rapid-MLX on the Mac
Start the server with a serve command
Send text or image requests
Read the text replies

Good to know:

• Size: about 13 GB on disk, against about 16 GB for a standard 4-bit build and about 51 GB for bf16
• Memory: measured peak use was about 14.6 to 16.5 GB
• Vision: it scores lower than the 4-bit build on MMBench-EN, because the mixed budget went to language
• License: Apache 2.0
• Cost to run: none named, no API key or account, only a Mac with enough memory
• Data it reaches: it downloads weights from Hugging Face, and the server runs on your own machine
• Not for: the source names no forbidden uses, and it is built for Apple Silicon, not Windows or Linux PCs

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: mlx
• Parameters: 27.4B
• Weights: about 109 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 715
• Likes: 10
• License: apache-2.0
• Languages: en, zh
• Based on: Qwen/Qwen3.8-27B
• Tags: safetensors, qwen3_5, apple-silicon, multimodal, qwen, quantized, mixed-precision, rapid-mlx, image-text-to-text, conversational

Links

• Model: https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX
• Files: https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX

Then follow the usage in its model card. Nothing in this kit runs by itself.