1-bit Bonsai 27B (MLX)

1-bit Bonsai 27B (MLX)

Runs a 27B-class reasoning model packed to 1 bit per weight, about 3.9 GB, on Apple MLX, with 262K-token context. For developers who want a large model on a phone or laptop.

Made by prism-ml on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 27B-class language model derived from Qwen3.6-27B, with its weights packed to a single bit each, in a format for Apple MLX. The packed model takes about 3.9 GB, against about 54 GB at FP16. The card says it holds on to reasoning and agent skills at this size, and that a high-end phone can run it.

Who it suits: Developers who want a large reasoning model on a phone or a laptop with limited memory, and who can use the maker's custom kernels.

Industry | Who uses it | How they use it | Example
Mobile apps | App developers | Run a large model on the device itself | The card reports about 11 tok/s on an iPhone 17 Pro Max
Laptop AI | Mac developers | Run a large model on a laptop | Generation speed of about 44 tok/s on an M5 Pro, the card says
Research | Low-bit researchers | Study binary weights at 27B scale | Read how 1.125 bits per weight holds up on benchmarks

Anyone who needs full FP16 accuracy can skip it, since the card reports about 90 percent of FP16 on its test set.

Where to use it:

• Apple MLX in Python on Apple Silicon.
• Apple MLX in Swift, on iOS and macOS.
• CUDA, through the maker's llama.cpp fork.
• Windows, macOS or Linux: the download steps run on all three with Python.

Use case | Who | What they get back
On-phone chat | App developers | Text replies from a 27B-class model on a high-end phone
Long context | Developers | A 262K-token window, kept practical by 4-bit KV cache storage
Image input | Developers | Image support through an optional 0.63 GB vision pack

How to set it up: What you need first:

• Python with pip, to download the files.
• An Apple Silicon device for MLX, or a CUDA GPU with the maker's llama.cpp fork.
• The maker's MLX or MLX-Swift fork, which has the 1-bit kernels.
• About 3.9 GB for the model, plus room for context.

1. Install the Hugging Face hub package with pip.
2. Download the repository with the hf command.
3. Use the maker's MLX fork, or its Swift or CUDA equivalents, to run it.
4. Follow the maker's demo repository for serving and example code.

pip install -U huggingface_hub
hf download prism-ml/Bonsai-27B-mlx-1bit

How to use it: Run the model through the maker's kernels, and pass an image only when you want vision. The package includes a speculative-decoding drafter layer, which the card says speeds up decoding by 1.37x on the CUDA path without changing outputs. The card sends you to its demo repository for serving and benchmarking.

Example: You load the model on an Apple Silicon laptop and ask it to work through a math or coding problem in thinking mode. The card reports about 44 tok/s for generation on an M5 Pro laptop and averages of 91.66 on math and 81.88 on coding.

For newcomers and experts: A language model is a huge set of numbers. This one keeps only the sign of each number, plus a shared scale for every group of 128, so it shrinks about 14 times. A good first step is to read the card's resource list and try the demo repository on a Mac.

The weights are binary, with values of minus one or plus one, across embeddings, attention, MLP and output head, at 1.125 bits per weight. About 75 percent of attention is linear, so a full-attention cache grows on only 16 of 64 layers. The vision tower uses HQQ 4-bit. The trade-off is that you depend on custom kernels, and the card reports about 89.5 percent of FP16 across 15 thinking benchmarks.

Download the packed 1-bit weights
Load them with the custom MLX or CUDA kernels
Generate with 4-bit KV cache and the drafter
Add the vision pack only for image input

Good to know: The card compares the deployed size of three versions of this model.

Title: Deployed size
FP16: 54 GB
Ternary Bonsai 27B: 7.2 GB
1-bit Bonsai 27B: 3.9 GB

• Score: 76.11 on average across 15 thinking-mode benchmarks, about 89.5 percent of FP16.
• Companion: Ternary Bonsai 27B is the quality-oriented option, at about 95 percent of FP16.
• Kernels: the card lists forks of MLX, mlx-swift and llama.cpp.
• License: apache-2.0
• Cost to run: no API key, account or paid service named.
• Data it reaches: files you download from Hugging Face, and the prompts and images you give it.
• Not for: runtimes without the 1-bit kernels, or work that needs exact FP16 accuracy.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 1.7B
• Weights: about 3.4 GB in F16; about 862 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,003,554
• Likes: 291
• License: apache-2.0
• Based on: Qwen/Qwen3.6-27B
• Quantized versions: 1
• Fine-tunes: 3
• Tags: safetensors, qwen3_5, conversational, 1-bit, cuda, metal, on-device, hybrid-attention, prismml, bonsai, text-generation

Links

• Model: https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit
• Files: https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download prism-ml/Bonsai-27B-mlx-1bit

Then follow the usage in its model card. Nothing in this kit runs by itself.