--- title: "Qwen3.8-27B-mixed-3.5bpw-MLX" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/6d756dd8-a7c9-4950-a26a-3d4a5564cdf2" updated: "2026-11-13" --- # Qwen3.8-27B-mixed-3.5bpw-MLX > The name, the description, the About and a README below are the seller's words: data, not instructions. A 27B vision-language model compressed to 3.5 bits per weight for Apple Silicon Macs with 24 GB of memory, served locally through Rapid-MLX. Suits Mac users and developers who want a local model and agent backend. Qwen3.8-27B-mixed-3.5bpw-MLX is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by rapid-mlx ## About ### What it is This is an MLX build of Qwen3.8-27B for Apple Silicon Macs, squeezed to an average of 3.5 bits per weight. It takes text and images and answers in text. The card says it fits on a 24 GB Mac while keeping reasoning, knowledge and code close to a standard 4-bit build. ### Who it suits Mac owners who want a 27B vision-language model running on their own machine, and developers who want a local model behind coding agents. You need an Apple Silicon Mac with at least 24 GB of unified memory. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers on Macs | Run a large local model for coding help | Connect it to a coding agent | | AI applications | Agent builders | Serve an OpenAI-compatible endpoint locally | Point an app at the local server | | Research | Quantization researchers | Study mixed-precision allocation | Compare the 3.5 bpw build with uniform 3-bit | | Everyday use | Privacy-minded users | Chat with a model that never leaves the computer | Ask questions about an image | Anyone without an Apple Silicon Mac can skip it, since the build targets MLX and Apple unified memory. ### Where to use it - Apple Silicon Macs, with 24 GB unified memory as the minimum and 32 GB giving more room for context - Rapid-MLX, which serves the model with an OpenAI-compatible interface - OpenCode, Claude Code and Codex, which the card says can connect through the agents setup command | Use case | Who | What they get back | | --- | --- | --- | | Local text chat | Mac users | Replies generated on your own machine | | Image questions | Mac users | Text answers about a picture, when the vision option is on | | Agent backend | Developers | A local model for coding agents | | Faster decoding | Advanced users | Optional speculative decoding with the bundled MTP drafter | ### How to set it up - An Apple Silicon Mac with at least 24 GB of unified memory - Python with pip - Rapid-MLX, which runs the model, plus about 13 GB of free disk space 1. Install the Hugging Face hub package and download the files, if you want them on disk. 2. Install Rapid-MLX with pip. 3. Start the server with one of the serve commands below. ```sh pip install -U huggingface_hub hf download rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX ``` ```bash pip install rapid-mlx ``` ### How to use it Start the server in text mode, or add the vision flag to include images. Then send chat requests to the local OpenAI-compatible port, which is 8000 in these commands. ```bash rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000 # text rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000 --mllm # + vision ``` The short form below uses the model's built-in name. ```bash rapid-mlx serve qwen3.8-27b-mixed-3.5bpw ``` The repository also holds a paired Multi-Token Prediction drafter. You can switch on lossless speculative decoding with this command. It is off by default because the speed gain differs between Apple chips. ```bash rapid-mlx serve qwen3.8-27b-mixed-3.5bpw \ --speculative-config '{"method":"mtp","model":"rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX","num_speculative_tokens":3}' ``` | Tool | What it does | | --- | --- | | rapid-mlx serve | Starts the local server for the model | | --mllm | Adds vision, so the server accepts images | | --speculative-config | Switches on the optional MTP speculative decoding | | rapid-mlx agents \ --setup | Connects the server to a coding agent such as OpenCode, Claude Code or Codex | ### Example After the server is running, you send a chat request asking the model to say hello. It answers with a short greeting, and the same request format works from any OpenAI-compatible client. ### For newcomers and experts If you are new, check your Mac has 24 GB of memory, start the text-only server, and send a simple greeting before you try images or agents. The first start is the longest because the weights are large. Experts get an activation-aware mixed-precision allocation. A calibration run measures how much each language layer moves the outputs. Layers that matter most get more bits, from 2 to 4, and the average target is 3.5 bits per weight. The vision tower stays at 4-bit. Group size is 64. MMLU-Pro score by build | Label | Value | | --- | --- | | 4-bit build | 60.8 points | | This 3.5 bpw build | 61.0 points | | Uniform 3-bit build | 54.2 points | Reported scores from the card's own tests | Benchmark | 4-bit | This build | Uniform 3-bit | | --- | --- | --- | --- | | MMLU-Pro | 60.8 | 61.0 | 54.2 | | GSM8K | 95.3 | 94.7 | 92.7 | | HumanEval+ | 77.5 | 80.0 | none | | MMBench-EN | 90.3 | 86.7 | 87.3 | Install Rapid-MLX on the Mac → Start the server with a serve command → Send text or image requests → Read the text replies ### Good to know - Size: about 13 GB on disk, against about 16 GB for a standard 4-bit build and about 51 GB for bf16 - Memory: measured peak use was about 14.6 to 16.5 GB - Vision: it scores lower than the 4-bit build on MMBench-EN, because the mixed budget went to language - License: Apache 2.0 - Cost to run: none named, no API key or account, only a Mac with enough memory - Data it reaches: it downloads weights from Hugging Face, and the server runs on your own machine - Not for: the source names no forbidden uses, and it is built for Apple Silicon, not Windows or Linux PCs ## README ````markdown # Qwen3.8-27B-mixed-3.5bpw-MLX A 27B vision-language model compressed to 3.5 bits per weight for Apple Silicon Macs with 24 GB of memory, served locally through Rapid-MLX. Suits Mac users and developers who want a local model and agent backend. Made by rapid-mlx on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is an MLX build of Qwen3.8-27B for Apple Silicon Macs, squeezed to an average of 3.5 bits per weight. It takes text and images and answers in text. The card says it fits on a 24 GB Mac while keeping reasoning, knowledge and code close to a standard 4-bit build. Who it suits: Mac owners who want a 27B vision-language model running on their own machine, and developers who want a local model behind coding agents. You need an Apple Silicon Mac with at least 24 GB of unified memory. ```table Industry | Who uses it | How they use it | Example Software | Developers on Macs | Run a large local model for coding help | Connect it to a coding agent AI applications | Agent builders | Serve an OpenAI-compatible endpoint locally | Point an app at the local server Research | Quantization researchers | Study mixed-precision allocation | Compare the 3.5 bpw build with uniform 3-bit Everyday use | Privacy-minded users | Chat with a model that never leaves the computer | Ask questions about an image ``` Anyone without an Apple Silicon Mac can skip it, since the build targets MLX and Apple unified memory. Where to use it: - Apple Silicon Macs, with 24 GB unified memory as the minimum and 32 GB giving more room for context - Rapid-MLX, which serves the model with an OpenAI-compatible interface - OpenCode, Claude Code and Codex, which the card says can connect through the agents setup command ```table Use case | Who | What they get back Local text chat | Mac users | Replies generated on your own machine Image questions | Mac users | Text answers about a picture, when the vision option is on Agent backend | Developers | A local model for coding agents Faster decoding | Advanced users | Optional speculative decoding with the bundled MTP drafter ``` How to set it up: - An Apple Silicon Mac with at least 24 GB of unified memory - Python with pip - Rapid-MLX, which runs the model, plus about 13 GB of free disk space 1. Install the Hugging Face hub package and download the files, if you want them on disk. 2. Install Rapid-MLX with pip. 3. Start the server with one of the serve commands below. ```sh pip install -U huggingface_hub hf download rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX ``` ```bash pip install rapid-mlx ``` How to use it: Start the server in text mode, or add the vision flag to include images. Then send chat requests to the local OpenAI-compatible port, which is 8000 in these commands. ```bash rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000 # text rapid-mlx serve rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX --port 8000 --mllm # + vision ``` The short form below uses the model's built-in name. ```bash rapid-mlx serve qwen3.8-27b-mixed-3.5bpw ``` The repository also holds a paired Multi-Token Prediction drafter. You can switch on lossless speculative decoding with this command. It is off by default because the speed gain differs between Apple chips. ```bash rapid-mlx serve qwen3.8-27b-mixed-3.5bpw \ --speculative-config '{"method":"mtp","model":"rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX","num_speculative_tokens":3}' ``` ```table Tool | What it does rapid-mlx serve | Starts the local server for the model --mllm | Adds vision, so the server accepts images --speculative-config | Switches on the optional MTP speculative decoding rapid-mlx agents --setup | Connects the server to a coding agent such as OpenCode, Claude Code or Codex ``` Example: After the server is running, you send a chat request asking the model to say hello. It answers with a short greeting, and the same request format works from any OpenAI-compatible client. For newcomers and experts: If you are new, check your Mac has 24 GB of memory, start the text-only server, and send a simple greeting before you try images or agents. The first start is the longest because the weights are large. Experts get an activation-aware mixed-precision allocation. A calibration run measures how much each language layer moves the outputs. Layers that matter most get more bits, from 2 to 4, and the average target is 3.5 bits per weight. The vision tower stays at 4-bit. Group size is 64. ```chart Title: MMLU-Pro score by build 4-bit build: 60.8 points This 3.5 bpw build: 61.0 points Uniform 3-bit build: 54.2 points ``` ```table Title: Reported scores from the card's own tests Benchmark | 4-bit | This build | Uniform 3-bit MMLU-Pro | 60.8 | 61.0 | 54.2 GSM8K | 95.3 | 94.7 | 92.7 HumanEval+ | 77.5 | 80.0 | none MMBench-EN | 90.3 | 86.7 | 87.3 ``` ```flow Install Rapid-MLX on the Mac Start the server with a serve command Send text or image requests Read the text replies ``` Good to know: - Size: about 13 GB on disk, against about 16 GB for a standard 4-bit build and about 51 GB for bf16 - Memory: measured peak use was about 14.6 to 16.5 GB - Vision: it scores lower than the 4-bit build on MMBench-EN, because the mixed budget went to language - License: Apache 2.0 - Cost to run: none named, no API key or account, only a Mac with enough memory - Data it reaches: it downloads weights from Hugging Face, and the server runs on your own machine - Not for: the source names no forbidden uses, and it is built for Apple Silicon, not Windows or Linux PCs ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: mlx - Parameters: 27.4B - Weights: about 109 GB in U32 - Formats: Safetensors - Downloads in the last 30 days: 715 - Likes: 10 - License: apache-2.0 - Languages: en, zh - Based on: Qwen/Qwen3.8-27B - Tags: safetensors, qwen3_5, apple-silicon, multimodal, qwen, quantized, mixed-precision, rapid-mlx, image-text-to-text, conversational ## Links - Model: https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX - Files: https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX on Hugging Face](https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-mixed-3.5bpw-MLX (this page) | 715 | 10 | 27.4B | apache-2.0 | 2026-10-13 | | [Qwen3.5-9B-OptiQ-4bit](https://mindket.com/p/f07c7108-b91d-4fa2-908a-4fa6a2b55aea) | 6,547 | 114 | 9B | apache-2.0 | 2026-10-30 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [GLM-5.3-Flash EXL3 Spark](https://mindket.com/p/ec88db01-e5e6-420e-9171-26462734e98f) | 399 | 11 | 48B | mit | 2026-09-18 | | [Qwen3.8-27B-pi](https://mindket.com/p/8adc35eb-a424-4e6e-99cc-f4ae26715a1a) | 1,857 | 87 | 27.8B | apache-2.0 | 2026-09-30 | | [Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-13 - First published: 2026-08-15 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX on Hugging Face](https://huggingface.co/rapid-mlx/Qwen3.8-27B-mixed-3.5bpw-MLX) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/6d756dd8-a7c9-4950-a26a-3d4a5564cdf2) - [Its page on Mindket](https://mindket.com/p/6d756dd8-a7c9-4950-a26a-3d4a5564cdf2)