Qwen3.8-Whittle-MoE-27B-MLX-4bit

Qwen3.8-Whittle-MoE-27B-MLX-4bit

A 4-bit MLX build for Apple Silicon of a mixture-of-experts model carved from the dense Qwen3.8-27B, with 17.8B active of 27B total parameters. For people who run local models with mlx-lm.

Made by barozp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 4-bit MLX conversion for Apple Silicon of Qwen3.8-Whittle-MoE-27B, which is itself a mixture-of-experts model cut out of the dense Qwen3.8-27B after training. Each layer holds 64 routed expert slivers and one shared expert, and the router picks 16 slivers per token. That gives 17.8B active parameters out of 27B total.

Who it suits: It suits people with an Apple Silicon Mac who run local models through mlx-lm and want to try an experimental mixture-of-experts variant of a dense model.

Industry | Who uses it | How they use it | Example
Local AI | Mac users | Chat with a 27B-class model on their own machine | Asking a general question in the terminal
Software | Developers | Run a local OpenAI-style server for apps and tests | Pointing a test script at the local server
Research | Model researchers | Study carving a dense model into experts and healing the router | Reading the loop-rate and knowledge numbers
Education | Students | Learn how mixture-of-experts layers are built from a dense layer | Following the source chain

If you are not on Apple Silicon, you can skip this build and look at the author's GGUF files instead.

Where to use it: You run it with mlx-lm on a Mac with Apple Silicon.

• A Mac with Apple Silicon, using the mlx-lm package
• A terminal, for the generate and server commands
• Windows, macOS or Linux with Python, only to download the files

Use case | Who | What they get back
Quick text chat | Mac users | A reply to a one-off prompt
Local API server | Developers | An OpenAI-compatible endpoint on your machine
Model comparison | Researchers | A 4-bit build to compare with the bf16 source

How to set it up: You need Python and the mlx-lm package on an Apple Silicon Mac. The model downloads by name the first time you run it.

• Python on an Apple Silicon Mac
• The mlx-lm package
• Enough unified memory for a 4-bit 27B model

1. Install mlx-lm.
2. Run the generate command to test it, or the server command to serve it.
3. Keep client prompts short at first while you check speed.

pip install -U mlx-lm

pip install -U huggingface_hub
hf download barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit

How to use it: Use mlx_lm.generate for a quick text chat, or mlx_lm.server for a local OpenAI-compatible server. This build has no MTP head, so it decodes one token at a time and MTPLX speculative decoding does not apply.

Tool | What it does
mlx_lm.generate | Answers one prompt from the command line
mlx_lm.server | Starts a local OpenAI-compatible server

python -m mlx_lm.generate \
  --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit \
  --prompt "Explain the Monty Hall problem briefly." \
  --max-tokens 512

python -m mlx_lm.server \
  --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit

Example: You run the generate command with a prompt asking for a short explanation of the Monty Hall problem and a limit of 512 new tokens. The model prints its explanation in the terminal and stops at the limit.

For newcomers and experts: A mixture-of-experts model sends each token to only a few small expert blocks, so fewer weights work per token. Start with the generate command and one short prompt, and see how it answers.

For experts, the dense feed-forward layer of width 17408 was split into 64 slivers of width 192 plus one shared expert of width 5120. Training only the routers lifted a knowledge test from 4 of 39 to 27 of 39. Version 2.1 added anti-loop training and a balancing epoch. The conversion used this command.

python -m mlx_lm.convert --hf-path logic65/Qwen3.8-Whittle-MoE-27B-A17.8B --mlx-path <out> --quantize --q-bits 4 --q-group-size 64

Dense Qwen3.8-27B is the starting point
Each layer is carved into 64 expert slivers
The routers are trained to heal the split
Anti-loop training is added in version 2.1
The result is converted to 4-bit MLX

Good to know: The authors report that single-turn loops fell from 69% to 8% and structured-output failures from about 75% to 22% in version 2.1. The knowledge test score is 28 of 39.

Title: Reported failure rates, before and after version 2.1
Loop rate before: 69%
Loop rate in v2.1: 8%
Structured-output failures before: 75%
Structured-output failures in v2.1: 22%

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the prompts you give it, run locally, and the model files you download
• Not for: speculative decoding with MTPLX, since the source has no MTP head, or machines that are not Apple Silicon

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx-lm
• Parameters: 26.9B
• Weights: about 108 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 1,611
• Likes: 11
• License: apache-2.0
• Based on: logic65/Qwen3.8-Whittle-MoE-27B-A17.8B
• Tags: mlx-lm, safetensors, qwen3_5_moe, qwen, qwen3, reasoning, opus-distill, whittle, expert-pruning, router-healing, quantized, text-generation

Links

• Model: https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit
• Files: https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit

Then follow the usage in its model card. Nothing in this kit runs by itself.