--- title: "Bonsai-27B-gguf" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/0f4c3b2e-9ef6-4680-896e-4319b270e9f8" updated: "2026-11-12" --- # Bonsai-27B-gguf > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 27B-class reasoning model as a 1-bit GGUF file of about 3.9 GB for llama.cpp, so it fits a laptop or one GPU. For people who want local reasoning on everyday hardware. Bonsai-27B-gguf is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by prism-ml ## About ### What it is Bonsai 27B is a 1-bit build of a 27B-class reasoning model, packed as a GGUF file for llama.cpp. It derives from Qwen3.6-27B and stores its language weights as single bits with a shared scale for each group. The deployed size is about 3.9 GB, against roughly 54 GB at FP16, so it can run on a laptop or one GPU. The authors say it keeps about 90 percent of FP16 quality. ### Who it suits It suits people who want a large reasoning model on everyday hardware and are comfortable running specialized llama.cpp builds. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI hobby | Enthusiasts | Run a 27B-class model on a laptop | Chatting with the model offline on a Mac using Metal | | Software development | Developers | Test a small-footprint reasoning model inside a local app | Serving it with llama.cpp on a single GPU | | Edge and mobile | App builders | Look at the MLX companion for Apple Silicon and iPhone | Trying the MLX version on a phone | | Model research | Researchers | Study 1-bit weights and speculative decoding | Comparing it with the ternary companion | You can skip it if your runtime cannot load the author's 1-bit kernels, since stock builds may not read this format. ### Where to use it - llama.cpp on CUDA, Metal or CPU, using the author's fork that holds the 1-bit kernels - Windows, macOS or Linux with Python, to download the files - Laptops and single GPUs, which the card names as its target - Apple Silicon and iPhone, through the separate MLX companion | Use case | Who | What they get back | | --- | --- | --- | | Local reasoning chat | Anyone with a laptop | Reasoning and coding answers from a file that fits in memory | | Image questions | Vision users | Answers about pictures, using the optional vision pack | | Faster decoding | GPU users | A lossless 1.37x speedup on CUDA from the bundled drafter layer | ### How to set it up You need Python for the download tool and a llama.cpp build that includes the 1-bit kernels. The weights take about 3.9 GB, plus 0.63 GB if you want image input. - Python on Windows, macOS or Linux - The huggingface\_hub package - The author's llama.cpp fork, linked from the card, built for CUDA, Metal or CPU - Room for the weights, plus about 4.3 GB of cache at the full 262K window 1. Install huggingface\_hub. 2. Download the model files. 3. Build the author's llama.cpp fork and point it at the GGUF file. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-gguf ``` ### How to use it The visible card gives the download step but no run command. It points to the author's demo repository for serving and benchmarking, and to a whitepaper for the method. Once your build loads the file, you chat with it like any local model. A drafter layer for speculative decoding ships with it. | Variant | Runs on | Size | | --- | --- | --- | | GGUF 1-bit, this repository | llama.cpp on CUDA, Metal or CPU | About 3.9 GB | | MLX 1-bit | Apple Silicon, including iPhone | Not stated | | Ternary GGUF | llama.cpp | About 7.2 GB | ### Example You load the GGUF in a build with the 1-bit kernels and ask a math or coding question. The model reasons through it and writes the answer. The authors measured roughly 44 tokens per second on an Apple M5 Pro laptop. ### For newcomers and experts If you are new to this, think of a huge model squeezed into a small file by storing each weight as a single bit. A first small try is the download step, then one short question once a compatible build is ready. For people who know the field, each weight is a sign bit and every group of 128 shares one FP16 scale, which comes to 1.125 bits per weight. Embeddings, attention, MLP and the LM head are all binary, and the vision tower uses 4-bit HQQ. A hybrid backbone with about 75 percent linear attention keeps the cache small. Size in GB | Label | Value | | --- | --- | | FP16 original | 54 GB | | Ternary Bonsai | 7.2 GB | | 1-bit Bonsai | 3.9 GB | You download the GGUF file → A build with 1-bit kernels loads the packed weights → The drafter layer proposes tokens ahead → The model checks them and writes the answer ### Good to know The quality figures come from the authors. Across 15 thinking-mode benchmarks they report an average of 76.11, which is 89.5 percent of FP16. - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you type and the downloaded files, plus images only if you load the vision pack - Not for: runtimes without the author's 1-bit kernels, or work that needs full FP16 quality ## README ````markdown # Bonsai-27B-gguf Runs a 27B-class reasoning model as a 1-bit GGUF file of about 3.9 GB for llama.cpp, so it fits a laptop or one GPU. For people who want local reasoning on everyday hardware. Made by prism-ml on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Bonsai 27B is a 1-bit build of a 27B-class reasoning model, packed as a GGUF file for llama.cpp. It derives from Qwen3.6-27B and stores its language weights as single bits with a shared scale for each group. The deployed size is about 3.9 GB, against roughly 54 GB at FP16, so it can run on a laptop or one GPU. The authors say it keeps about 90 percent of FP16 quality. Who it suits: It suits people who want a large reasoning model on everyday hardware and are comfortable running specialized llama.cpp builds. ```table Industry | Who uses it | How they use it | Example Local AI hobby | Enthusiasts | Run a 27B-class model on a laptop | Chatting with the model offline on a Mac using Metal Software development | Developers | Test a small-footprint reasoning model inside a local app | Serving it with llama.cpp on a single GPU Edge and mobile | App builders | Look at the MLX companion for Apple Silicon and iPhone | Trying the MLX version on a phone Model research | Researchers | Study 1-bit weights and speculative decoding | Comparing it with the ternary companion ``` You can skip it if your runtime cannot load the author's 1-bit kernels, since stock builds may not read this format. Where to use it: - llama.cpp on CUDA, Metal or CPU, using the author's fork that holds the 1-bit kernels - Windows, macOS or Linux with Python, to download the files - Laptops and single GPUs, which the card names as its target - Apple Silicon and iPhone, through the separate MLX companion ```table Use case | Who | What they get back Local reasoning chat | Anyone with a laptop | Reasoning and coding answers from a file that fits in memory Image questions | Vision users | Answers about pictures, using the optional vision pack Faster decoding | GPU users | A lossless 1.37x speedup on CUDA from the bundled drafter layer ``` How to set it up: You need Python for the download tool and a llama.cpp build that includes the 1-bit kernels. The weights take about 3.9 GB, plus 0.63 GB if you want image input. - Python on Windows, macOS or Linux - The huggingface_hub package - The author's llama.cpp fork, linked from the card, built for CUDA, Metal or CPU - Room for the weights, plus about 4.3 GB of cache at the full 262K window 1. Install huggingface_hub. 2. Download the model files. 3. Build the author's llama.cpp fork and point it at the GGUF file. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-gguf ``` How to use it: The visible card gives the download step but no run command. It points to the author's demo repository for serving and benchmarking, and to a whitepaper for the method. Once your build loads the file, you chat with it like any local model. A drafter layer for speculative decoding ships with it. ```table Variant | Runs on | Size GGUF 1-bit, this repository | llama.cpp on CUDA, Metal or CPU | About 3.9 GB MLX 1-bit | Apple Silicon, including iPhone | Not stated Ternary GGUF | llama.cpp | About 7.2 GB ``` Example: You load the GGUF in a build with the 1-bit kernels and ask a math or coding question. The model reasons through it and writes the answer. The authors measured roughly 44 tokens per second on an Apple M5 Pro laptop. For newcomers and experts: If you are new to this, think of a huge model squeezed into a small file by storing each weight as a single bit. A first small try is the download step, then one short question once a compatible build is ready. For people who know the field, each weight is a sign bit and every group of 128 shares one FP16 scale, which comes to 1.125 bits per weight. Embeddings, attention, MLP and the LM head are all binary, and the vision tower uses 4-bit HQQ. A hybrid backbone with about 75 percent linear attention keeps the cache small. ```chart Title: Size in GB FP16 original: 54 GB Ternary Bonsai: 7.2 GB 1-bit Bonsai: 3.9 GB ``` ```flow You download the GGUF file A build with 1-bit kernels loads the packed weights The drafter layer proposes tokens ahead The model checks them and writes the answer ``` Good to know: The quality figures come from the authors. Across 15 thinking-mode benchmarks they report an average of 76.11, which is 89.5 percent of FP16. - License: Apache 2.0 - Cost to run: none named - Data it reaches: the prompts you type and the downloaded files, plus images only if you load the vision pack - Not for: runtimes without the author's 1-bit kernels, or work that needs full FP16 quality ## On Hugging Face, 13 November 2026 - Task: text generation - Library: llama.cpp - Parameters: 26.9B - Weights: about 13.4 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 179,818 - Likes: 945 - License: apache-2.0 - Based on: Qwen/Qwen3.6-27B - Quantized versions: 8 - Fine-tunes: 7 - Tags: llama.cpp, gguf, conversational, 1-bit, llama-cpp, cuda, metal, on-device, hybrid-attention, prismml, bonsai, text-generation ## Links - Model: https://huggingface.co/prism-ml/Bonsai-27B-gguf - Files: https://huggingface.co/prism-ml/Bonsai-27B-gguf/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-27B-gguf ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [prism-ml/Bonsai-27B-gguf on Hugging Face](https://huggingface.co/prism-ml/Bonsai-27B-gguf) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Bonsai-27B-gguf (this page) | 179,818 | 945 | 26.9B | apache-2.0 | 2026-07-17 | | [Bonsai-1.7B-GGUF](https://mindket.com/p/976ed9c8-c475-4a88-a05b-e149421d5c28) | 173,963 | 100 | 1.7B | apache-2.0 | 2026-04-18 | | [1-bit Bonsai 27B (MLX)](https://mindket.com/p/d7dcb13a-512c-4d6e-8d42-d76b91e7140d) | 1,003,554 | 291 | 1.7B | apache-2.0 | 2026-07-14 | | [Bonsai-1.7B-unpacked](https://mindket.com/p/14bbc17d-6c93-4d4f-a6a2-1bb2c0327eb6) | 2,362 | 15 | 1.7B | apache-2.0 | 2026-04-16 | | [MNN-Pocket-0.3B-GGUF](https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb) | 12,467 | 20 | 362M | apache-2.0 | 2026-11-10 | | [MiniCPM5-2B-GGUF](https://mindket.com/p/47f50d35-3645-40b1-914d-63bedd57f448) | 481,119 | 393 | 2.5B | apache-2.0 | 2026-11-11 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · walrus01 · 2026-07-27](https://news.ycombinator.com/item?id=49067227): Kimi-K3 Releases on HuggingFace 7/27: Recently spent a few hours messing with bonsai 27B and ternary bonsai 27B at total weights + context fitting in slightly under 6GB RAM, and it's just dumb as hell. It writes what seems like grammatically correct content but it's extremely limited. It will also happily hallucinate new names and… - [Bluesky · crumb · 2026-07-16](https://bsky.app/profile/crumb.bsky.social/post/3mqrwi6yac226): the binary 27b from prismml actually runs really really well on my laptop and seems entirely usable for synth data workloads and possibly function search. \<4gb. pleasantly surprised. huggingface.co/prism-ml/Bon... - [Hacker News · meatmanek · 2026-07-21](https://news.ycombinator.com/item?id=48996530): How to pack ternary numbers in 8-bit bytes: There are 1-bit\* LLMs (+1/-1, usually), though they often come with an additional scaling parameter: https://huggingface.co/prism-ml/Bonsai-27B-gguf\#weight-repre... \> Weight Representation: Q1\_0\_g128 Each weight is a single sign bit: 0 maps to −scale, 1 maps to +scale. Every group of 128 weights… - [Bluesky · Outerspace79 · 2026-09-07](https://bsky.app/profile/outerspace79.bsky.social/post/3muwk2cfyy22z): W teorii możesz popróbować modeli 1-bit od prism. 27b a waży tylko 7GB. Może wlezie i pojedzie… a nóż się do czegoś przyda. LMF2.5 za to są dedykowane do CPU i faktycznie są szybkie. Może Ci do czegoś Ci podpasują. huggingface.co/prism-ml/Ter... - [Bluesky · NoizBuster · 2026-07-15](https://bsky.app/profile/noizbuster.bsky.social/post/3mqnvzys5wk2u): PrismML 이 차력쇼를 계속 하고 있었네요 ㅋㅋ 원본은 qwen 27B 입니다. dense 모델 좋아하시는 분들 보조용으로 로컬에서 돌리시기 좋아보입니다. 당분간은 게임용 GPU 한장으로 돌리기에 이거보다 좋은게 있을까 싶네요 huggingface.co/prism-ml/Ter... - [Bluesky · probbrain.com · 2026-07-15](https://bsky.app/profile/probbrain.bsky.social/post/3mqntifmrzy2x): prism-ml/Bonsai-27B-gguf · 124♥ · 513 dl - [Bluesky · probbrain.com · 2026-07-14](https://bsky.app/profile/probbrain.bsky.social/post/3mqnfa2yjqo2b): prism-ml/Ternary-Bonsai-27B-gguf · 116♥ · 23 dl ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-07-17 - First published: 2026-07-04 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open prism-ml/Bonsai-27B-gguf on Hugging Face](https://huggingface.co/prism-ml/Bonsai-27B-gguf) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/0f4c3b2e-9ef6-4680-896e-4319b270e9f8) - [Its page on Mindket](https://mindket.com/p/0f4c3b2e-9ef6-4680-896e-4319b270e9f8)