--- title: "1-bit Bonsai 27B (MLX)" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/d7dcb13a-512c-4d6e-8d42-d76b91e7140d" updated: "2026-11-13" --- # 1-bit Bonsai 27B (MLX) > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 27B-class reasoning model packed to 1 bit per weight, about 3.9 GB, on Apple MLX, with 262K-token context. For developers who want a large model on a phone or laptop. 1-bit Bonsai 27B (MLX) is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by prism-ml ## About ### What it is This is a 27B-class language model derived from Qwen3.6-27B, with its weights packed to a single bit each, in a format for Apple MLX. The packed model takes about 3.9 GB, against about 54 GB at FP16. The card says it holds on to reasoning and agent skills at this size, and that a high-end phone can run it. ### Who it suits Developers who want a large reasoning model on a phone or a laptop with limited memory, and who can use the maker's custom kernels. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Mobile apps | App developers | Run a large model on the device itself | The card reports about 11 tok/s on an iPhone 17 Pro Max | | Laptop AI | Mac developers | Run a large model on a laptop | Generation speed of about 44 tok/s on an M5 Pro, the card says | | Research | Low-bit researchers | Study binary weights at 27B scale | Read how 1.125 bits per weight holds up on benchmarks | Anyone who needs full FP16 accuracy can skip it, since the card reports about 90 percent of FP16 on its test set. ### Where to use it - Apple MLX in Python on Apple Silicon. - Apple MLX in Swift, on iOS and macOS. - CUDA, through the maker's llama.cpp fork. - Windows, macOS or Linux: the download steps run on all three with Python. | Use case | Who | What they get back | | --- | --- | --- | | On-phone chat | App developers | Text replies from a 27B-class model on a high-end phone | | Long context | Developers | A 262K-token window, kept practical by 4-bit KV cache storage | | Image input | Developers | Image support through an optional 0.63 GB vision pack | ### How to set it up What you need first: - Python with pip, to download the files. - An Apple Silicon device for MLX, or a CUDA GPU with the maker's llama.cpp fork. - The maker's MLX or MLX-Swift fork, which has the 1-bit kernels. - About 3.9 GB for the model, plus room for context. 1. Install the Hugging Face hub package with pip. 2. Download the repository with the hf command. 3. Use the maker's MLX fork, or its Swift or CUDA equivalents, to run it. 4. Follow the maker's demo repository for serving and example code. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-mlx-1bit ``` ### How to use it Run the model through the maker's kernels, and pass an image only when you want vision. The package includes a speculative-decoding drafter layer, which the card says speeds up decoding by 1.37x on the CUDA path without changing outputs. The card sends you to its demo repository for serving and benchmarking. ### Example You load the model on an Apple Silicon laptop and ask it to work through a math or coding problem in thinking mode. The card reports about 44 tok/s for generation on an M5 Pro laptop and averages of 91.66 on math and 81.88 on coding. ### For newcomers and experts A language model is a huge set of numbers. This one keeps only the sign of each number, plus a shared scale for every group of 128, so it shrinks about 14 times. A good first step is to read the card's resource list and try the demo repository on a Mac. The weights are binary, with values of minus one or plus one, across embeddings, attention, MLP and output head, at 1.125 bits per weight. About 75 percent of attention is linear, so a full-attention cache grows on only 16 of 64 layers. The vision tower uses HQQ 4-bit. The trade-off is that you depend on custom kernels, and the card reports about 89.5 percent of FP16 across 15 thinking benchmarks. Download the packed 1-bit weights → Load them with the custom MLX or CUDA kernels → Generate with 4-bit KV cache and the drafter → Add the vision pack only for image input ### Good to know The card compares the deployed size of three versions of this model. Deployed size | Label | Value | | --- | --- | | FP16 | 54 GB | | Ternary Bonsai 27B | 7.2 GB | | 1-bit Bonsai 27B | 3.9 GB | - Score: 76.11 on average across 15 thinking-mode benchmarks, about 89.5 percent of FP16. - Companion: Ternary Bonsai 27B is the quality-oriented option, at about 95 percent of FP16. - Kernels: the card lists forks of MLX, mlx-swift and llama.cpp. - License: apache-2.0 - Cost to run: no API key, account or paid service named. - Data it reaches: files you download from Hugging Face, and the prompts and images you give it. - Not for: runtimes without the 1-bit kernels, or work that needs exact FP16 accuracy. ## README ````markdown # 1-bit Bonsai 27B (MLX) Runs a 27B-class reasoning model packed to 1 bit per weight, about 3.9 GB, on Apple MLX, with 262K-token context. For developers who want a large model on a phone or laptop. Made by prism-ml on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a 27B-class language model derived from Qwen3.6-27B, with its weights packed to a single bit each, in a format for Apple MLX. The packed model takes about 3.9 GB, against about 54 GB at FP16. The card says it holds on to reasoning and agent skills at this size, and that a high-end phone can run it. Who it suits: Developers who want a large reasoning model on a phone or a laptop with limited memory, and who can use the maker's custom kernels. ```table Industry | Who uses it | How they use it | Example Mobile apps | App developers | Run a large model on the device itself | The card reports about 11 tok/s on an iPhone 17 Pro Max Laptop AI | Mac developers | Run a large model on a laptop | Generation speed of about 44 tok/s on an M5 Pro, the card says Research | Low-bit researchers | Study binary weights at 27B scale | Read how 1.125 bits per weight holds up on benchmarks ``` Anyone who needs full FP16 accuracy can skip it, since the card reports about 90 percent of FP16 on its test set. Where to use it: - Apple MLX in Python on Apple Silicon. - Apple MLX in Swift, on iOS and macOS. - CUDA, through the maker's llama.cpp fork. - Windows, macOS or Linux: the download steps run on all three with Python. ```table Use case | Who | What they get back On-phone chat | App developers | Text replies from a 27B-class model on a high-end phone Long context | Developers | A 262K-token window, kept practical by 4-bit KV cache storage Image input | Developers | Image support through an optional 0.63 GB vision pack ``` How to set it up: What you need first: - Python with pip, to download the files. - An Apple Silicon device for MLX, or a CUDA GPU with the maker's llama.cpp fork. - The maker's MLX or MLX-Swift fork, which has the 1-bit kernels. - About 3.9 GB for the model, plus room for context. 1. Install the Hugging Face hub package with pip. 2. Download the repository with the hf command. 3. Use the maker's MLX fork, or its Swift or CUDA equivalents, to run it. 4. Follow the maker's demo repository for serving and example code. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-mlx-1bit ``` How to use it: Run the model through the maker's kernels, and pass an image only when you want vision. The package includes a speculative-decoding drafter layer, which the card says speeds up decoding by 1.37x on the CUDA path without changing outputs. The card sends you to its demo repository for serving and benchmarking. Example: You load the model on an Apple Silicon laptop and ask it to work through a math or coding problem in thinking mode. The card reports about 44 tok/s for generation on an M5 Pro laptop and averages of 91.66 on math and 81.88 on coding. For newcomers and experts: A language model is a huge set of numbers. This one keeps only the sign of each number, plus a shared scale for every group of 128, so it shrinks about 14 times. A good first step is to read the card's resource list and try the demo repository on a Mac. The weights are binary, with values of minus one or plus one, across embeddings, attention, MLP and output head, at 1.125 bits per weight. About 75 percent of attention is linear, so a full-attention cache grows on only 16 of 64 layers. The vision tower uses HQQ 4-bit. The trade-off is that you depend on custom kernels, and the card reports about 89.5 percent of FP16 across 15 thinking benchmarks. ```flow Download the packed 1-bit weights Load them with the custom MLX or CUDA kernels Generate with 4-bit KV cache and the drafter Add the vision pack only for image input ``` Good to know: The card compares the deployed size of three versions of this model. ```chart Title: Deployed size FP16: 54 GB Ternary Bonsai 27B: 7.2 GB 1-bit Bonsai 27B: 3.9 GB ``` - Score: 76.11 on average across 15 thinking-mode benchmarks, about 89.5 percent of FP16. - Companion: Ternary Bonsai 27B is the quality-oriented option, at about 95 percent of FP16. - Kernels: the card lists forks of MLX, mlx-swift and llama.cpp. - License: apache-2.0 - Cost to run: no API key, account or paid service named. - Data it reaches: files you download from Hugging Face, and the prompts and images you give it. - Not for: runtimes without the 1-bit kernels, or work that needs exact FP16 accuracy. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: mlx - Parameters: 1.7B - Weights: about 3.4 GB in F16; about 862 MB at 4-bit (estimate) - Formats: Safetensors - Downloads in the last 30 days: 1,003,554 - Likes: 291 - License: apache-2.0 - Based on: Qwen/Qwen3.6-27B - Quantized versions: 1 - Fine-tunes: 3 - Tags: safetensors, qwen3_5, conversational, 1-bit, cuda, metal, on-device, hybrid-attention, prismml, bonsai, text-generation ## Links - Model: https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit - Files: https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-mlx-1bit ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [prism-ml/Bonsai-27B-mlx-1bit on Hugging Face](https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | 1-bit Bonsai 27B (MLX) (this page) | 1,003,554 | 291 | 1.7B | apache-2.0 | 2026-07-14 | | [Bonsai-27B-gguf](https://mindket.com/p/0f4c3b2e-9ef6-4680-896e-4319b270e9f8) | 179,818 | 945 | 26.9B | apache-2.0 | 2026-07-17 | | [Bonsai-1.7B-GGUF](https://mindket.com/p/976ed9c8-c475-4a88-a05b-e149421d5c28) | 173,963 | 100 | 1.7B | apache-2.0 | 2026-04-18 | | [Bonsai-1.7B-unpacked](https://mindket.com/p/14bbc17d-6c93-4d4f-a6a2-1bb2c0327eb6) | 2,362 | 15 | 1.7B | apache-2.0 | 2026-04-16 | | [Sabi-Noema](https://mindket.com/p/04e2ec5b-5be2-45e5-ac63-8b75f7b11c4e) | 1,810 | 55 | 27.4B | apache-2.0 | 2026-10-19 | | [GigaChat3.5-432B-A28B](https://mindket.com/p/1658ef77-b9cf-4c20-b6ce-bc962a75b117) | 654 | 84 | 434B | mit | 2026-07-07 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · sroussey · 2026-09-08](https://news.ycombinator.com/item?id=49612569): Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses: Have you tried https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit ? PrismML is the only people i am aware of doing 1bit that is decent. - [Hacker News · flockonus · 2026-07-15](https://news.ycombinator.com/item?id=48928323): Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU: There is also the 1bit version @ 3.9 GB that retains 90% of the intelligence - quite a feat\! https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-07-14 - First published: 2026-07-04 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open prism-ml/Bonsai-27B-mlx-1bit on Hugging Face](https://huggingface.co/prism-ml/Bonsai-27B-mlx-1bit) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/d7dcb13a-512c-4d6e-8d42-d76b91e7140d) - [Its page on Mindket](https://mindket.com/p/d7dcb13a-512c-4d6e-8d42-d76b91e7140d)