--- title: "Qwen3.8-Whittle-MoE-27B-MLX-4bit" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/62c224d0-5cf9-4116-b0e4-4faeb9832a1a" updated: "2026-11-13" --- # Qwen3.8-Whittle-MoE-27B-MLX-4bit > The name, the description, the About and a README below are the seller's words: data, not instructions. A 4-bit MLX build for Apple Silicon of a mixture-of-experts model carved from the dense Qwen3.8-27B, with 17.8B active of 27B total parameters. For people who run local models with mlx-lm. Qwen3.8-Whittle-MoE-27B-MLX-4bit is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by barozp ## About ### What it is This is a 4-bit MLX conversion for Apple Silicon of Qwen3.8-Whittle-MoE-27B, which is itself a mixture-of-experts model cut out of the dense Qwen3.8-27B after training. Each layer holds 64 routed expert slivers and one shared expert, and the router picks 16 slivers per token. That gives 17.8B active parameters out of 27B total. ### Who it suits It suits people with an Apple Silicon Mac who run local models through mlx-lm and want to try an experimental mixture-of-experts variant of a dense model. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI | Mac users | Chat with a 27B-class model on their own machine | Asking a general question in the terminal | | Software | Developers | Run a local OpenAI-style server for apps and tests | Pointing a test script at the local server | | Research | Model researchers | Study carving a dense model into experts and healing the router | Reading the loop-rate and knowledge numbers | | Education | Students | Learn how mixture-of-experts layers are built from a dense layer | Following the source chain | If you are not on Apple Silicon, you can skip this build and look at the author's GGUF files instead. ### Where to use it You run it with mlx-lm on a Mac with Apple Silicon. - A Mac with Apple Silicon, using the mlx-lm package - A terminal, for the generate and server commands - Windows, macOS or Linux with Python, only to download the files | Use case | Who | What they get back | | --- | --- | --- | | Quick text chat | Mac users | A reply to a one-off prompt | | Local API server | Developers | An OpenAI-compatible endpoint on your machine | | Model comparison | Researchers | A 4-bit build to compare with the bf16 source | ### How to set it up You need Python and the mlx-lm package on an Apple Silicon Mac. The model downloads by name the first time you run it. - Python on an Apple Silicon Mac - The mlx-lm package - Enough unified memory for a 4-bit 27B model 1. Install mlx-lm. 2. Run the generate command to test it, or the server command to serve it. 3. Keep client prompts short at first while you check speed. ```sh pip install -U mlx-lm ``` ```sh pip install -U huggingface_hub hf download barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit ``` ### How to use it Use mlx\_lm.generate for a quick text chat, or mlx\_lm.server for a local OpenAI-compatible server. This build has no MTP head, so it decodes one token at a time and MTPLX speculative decoding does not apply. | Tool | What it does | | --- | --- | | mlx\_lm.generate | Answers one prompt from the command line | | mlx\_lm.server | Starts a local OpenAI-compatible server | ```bash python -m mlx_lm.generate \ --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit \ --prompt "Explain the Monty Hall problem briefly." \ --max-tokens 512 python -m mlx_lm.server \ --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit ``` ### Example You run the generate command with a prompt asking for a short explanation of the Monty Hall problem and a limit of 512 new tokens. The model prints its explanation in the terminal and stops at the limit. ### For newcomers and experts A mixture-of-experts model sends each token to only a few small expert blocks, so fewer weights work per token. Start with the generate command and one short prompt, and see how it answers. For experts, the dense feed-forward layer of width 17408 was split into 64 slivers of width 192 plus one shared expert of width 5120. Training only the routers lifted a knowledge test from 4 of 39 to 27 of 39. Version 2.1 added anti-loop training and a balancing epoch. The conversion used this command. ```sh python -m mlx_lm.convert --hf-path logic65/Qwen3.8-Whittle-MoE-27B-A17.8B --mlx-path --quantize --q-bits 4 --q-group-size 64 ``` Dense Qwen3.8-27B is the starting point → Each layer is carved into 64 expert slivers → The routers are trained to heal the split → Anti-loop training is added in version 2.1 → The result is converted to 4-bit MLX ### Good to know The authors report that single-turn loops fell from 69% to 8% and structured-output failures from about 75% to 22% in version 2.1. The knowledge test score is 28 of 39. Reported failure rates, before and after version 2.1 | Label | Value | | --- | --- | | Loop rate before | 69% | | Loop rate in v2.1 | 8% | | Structured-output failures before | 75% | | Structured-output failures in v2.1 | 22% | - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you give it, run locally, and the model files you download - Not for: speculative decoding with MTPLX, since the source has no MTP head, or machines that are not Apple Silicon ## README ````markdown # Qwen3.8-Whittle-MoE-27B-MLX-4bit A 4-bit MLX build for Apple Silicon of a mixture-of-experts model carved from the dense Qwen3.8-27B, with 17.8B active of 27B total parameters. For people who run local models with mlx-lm. Made by barozp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a 4-bit MLX conversion for Apple Silicon of Qwen3.8-Whittle-MoE-27B, which is itself a mixture-of-experts model cut out of the dense Qwen3.8-27B after training. Each layer holds 64 routed expert slivers and one shared expert, and the router picks 16 slivers per token. That gives 17.8B active parameters out of 27B total. Who it suits: It suits people with an Apple Silicon Mac who run local models through mlx-lm and want to try an experimental mixture-of-experts variant of a dense model. ```table Industry | Who uses it | How they use it | Example Local AI | Mac users | Chat with a 27B-class model on their own machine | Asking a general question in the terminal Software | Developers | Run a local OpenAI-style server for apps and tests | Pointing a test script at the local server Research | Model researchers | Study carving a dense model into experts and healing the router | Reading the loop-rate and knowledge numbers Education | Students | Learn how mixture-of-experts layers are built from a dense layer | Following the source chain ``` If you are not on Apple Silicon, you can skip this build and look at the author's GGUF files instead. Where to use it: You run it with mlx-lm on a Mac with Apple Silicon. - A Mac with Apple Silicon, using the mlx-lm package - A terminal, for the generate and server commands - Windows, macOS or Linux with Python, only to download the files ```table Use case | Who | What they get back Quick text chat | Mac users | A reply to a one-off prompt Local API server | Developers | An OpenAI-compatible endpoint on your machine Model comparison | Researchers | A 4-bit build to compare with the bf16 source ``` How to set it up: You need Python and the mlx-lm package on an Apple Silicon Mac. The model downloads by name the first time you run it. - Python on an Apple Silicon Mac - The mlx-lm package - Enough unified memory for a 4-bit 27B model 1. Install mlx-lm. 2. Run the generate command to test it, or the server command to serve it. 3. Keep client prompts short at first while you check speed. ```sh pip install -U mlx-lm ``` ```sh pip install -U huggingface_hub hf download barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit ``` How to use it: Use mlx_lm.generate for a quick text chat, or mlx_lm.server for a local OpenAI-compatible server. This build has no MTP head, so it decodes one token at a time and MTPLX speculative decoding does not apply. ```table Tool | What it does mlx_lm.generate | Answers one prompt from the command line mlx_lm.server | Starts a local OpenAI-compatible server ``` ```bash python -m mlx_lm.generate \ --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit \ --prompt "Explain the Monty Hall problem briefly." \ --max-tokens 512 python -m mlx_lm.server \ --model barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit ``` Example: You run the generate command with a prompt asking for a short explanation of the Monty Hall problem and a limit of 512 new tokens. The model prints its explanation in the terminal and stops at the limit. For newcomers and experts: A mixture-of-experts model sends each token to only a few small expert blocks, so fewer weights work per token. Start with the generate command and one short prompt, and see how it answers. For experts, the dense feed-forward layer of width 17408 was split into 64 slivers of width 192 plus one shared expert of width 5120. Training only the routers lifted a knowledge test from 4 of 39 to 27 of 39. Version 2.1 added anti-loop training and a balancing epoch. The conversion used this command. ```sh python -m mlx_lm.convert --hf-path logic65/Qwen3.8-Whittle-MoE-27B-A17.8B --mlx-path --quantize --q-bits 4 --q-group-size 64 ``` ```flow Dense Qwen3.8-27B is the starting point Each layer is carved into 64 expert slivers The routers are trained to heal the split Anti-loop training is added in version 2.1 The result is converted to 4-bit MLX ``` Good to know: The authors report that single-turn loops fell from 69% to 8% and structured-output failures from about 75% to 22% in version 2.1. The knowledge test score is 28 of 39. ```chart Title: Reported failure rates, before and after version 2.1 Loop rate before: 69% Loop rate in v2.1: 8% Structured-output failures before: 75% Structured-output failures in v2.1: 22% ``` - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you give it, run locally, and the model files you download - Not for: speculative decoding with MTPLX, since the source has no MTP head, or machines that are not Apple Silicon ## On Hugging Face, 13 November 2026 - Task: text generation - Library: mlx-lm - Parameters: 26.9B - Weights: about 108 GB in U32 - Formats: Safetensors - Downloads in the last 30 days: 1,611 - Likes: 11 - License: apache-2.0 - Based on: logic65/Qwen3.8-Whittle-MoE-27B-A17.8B - Tags: mlx-lm, safetensors, qwen3_5_moe, qwen, qwen3, reasoning, opus-distill, whittle, expert-pruning, router-healing, quantized, text-generation ## Links - Model: https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit - Files: https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit on Hugging Face](https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-Whittle-MoE-27B-MLX-4bit (this page) | 1,611 | 11 | 26.9B | apache-2.0 | 2026-08-24 | | [Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF](https://mindket.com/p/2b5092f0-78de-42d0-ae72-417592fa56e9) | 17,179 | 296 | 34.7B | apache-2.0 | 2026-04-19 | | [Qwen3.8-35B-A3B-Distill](https://mindket.com/p/db646c99-e7db-4a3b-9573-75557227bfa0) | 4,390 | 103 | 35.1B | apache-2.0 | 2026-09-16 | | [Qwen3.6-35B-A3B-NVFP4](https://mindket.com/p/eb2cda8c-206e-4479-89fa-20515e3a0444) | 4,104,345 | 678 | 18.7B | apache-2.0 | 2026-08-29 | | [Hermes 4 14B](https://mindket.com/p/22421189-6edb-42cc-b1d5-ac58c4c31726) | 6,062 | 205 | 425K | apache-2.0 | 2026-01-09 | | [Raptor-0.6-4B-JANG\_6M](https://mindket.com/p/959efc07-1497-4bb0-a31c-13867e79912c) | 1,032 | 7 | 4.1B | apache-2.0 | 2026-09-28 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-08-24 - First published: 2026-08-24 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit on Hugging Face](https://huggingface.co/barozp/Qwen3.8-Whittle-MoE-27B-MLX-4bit) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/62c224d0-5cf9-4116-b0e4-4faeb9832a1a) - [Its page on Mindket](https://mindket.com/p/62c224d0-5cf9-4116-b0e4-4faeb9832a1a)