--- title: "Bonsai-1.7B-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/976ed9c8-c475-4a88-a05b-e149421d5c28" updated: "2026-11-13" --- # Bonsai-1.7B-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Generates text with a 1.7B language model packed to 1 bit per weight, about 0.24 GB, for llama.cpp on CUDA, Metal and CPU. For developers who need a tiny local model. Bonsai-1.7B-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by prism-ml ## About ### What it is Bonsai-1.7B is a small text generation model where every weight, including embeddings and the output head, is stored in a single bit. It uses the Qwen3-1.7B design, packed in the GGUF Q1\_0 format. The weights take about 0.24 GB, against 3.44 GB at FP16, and it runs in llama.cpp on CUDA, Metal and CPU. ### Who it suits Developers who need a language model on a small device or with little memory, and who are happy to use the maker's fork of llama.cpp. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Mobile apps | App developers | Run a small model on a phone or tablet | The card lists Swift for iPhone and iPad, and Android | | Edge computing | Embedded engineers | Fit a language model into tight memory | Load a 0.24 GB model on a small machine | | Hobby projects | Local AI tinkerers | Try 1-bit models on a laptop | Chat with the model through llama.cpp on a Mac | Anyone who needs the best answer quality can skip it, since a 1.7B model at 1 bit trades quality for size. ### Where to use it - llama.cpp on CUDA GPUs, including RTX and datacenter cards. - llama.cpp on Metal, for Macs. - llama.cpp on CPU. - Swift on iPhone and iPad, and Android. - Windows, macOS or Linux: the download steps run on all three with Python. | Use case | Who | What they get back | | --- | --- | --- | | On-device chat | App developers | Text replies from a model of about 0.24 GB | | Low-memory servers | Engineers | A language model that loads in a fraction of the FP16 memory | | Speed tests | Tinkerers | The card reports up to 3.0x faster than FP16 on an RTX 4090 | ### How to set it up What you need first: - Python with pip, to download the file. - The PrismML fork of llama.cpp, which includes the Q1\_0 kernels the model needs. - A CUDA GPU, a Mac with Metal, or a CPU. 1. Install the Hugging Face hub package with pip. 2. Download this repository with the hf command. 3. Build the PrismML fork of llama.cpp for your hardware, as the card describes. 4. Run the GGUF file with that build. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-1.7B-gguf ``` ### How to use it Run the file with the fork's llama.cpp tools. A short system prompt works, and the card suggests the sampling settings below. ```text You are a helpful assistant ``` Suggested generation settings | Parameter | Default | Suggested range | | --- | --- | --- | | Temperature | 0.5 | 0.5 to 0.7 | | Top-k | 20 | 20 to 40 | | Top-p | 0.9 | 0.85 to 0.95 | | Repetition penalty | 1.0 | none given | ### Example You start the model in the fork's llama.cpp with the system prompt above and ask a short question. It returns a plain text answer, generated from a file under 0.3 GB. ### For newcomers and experts Most language models store each number in 16 bits. This one stores each in 1 bit, which is why the file is so small. A good first step is to download the file and chat with it in the fork of llama.cpp. In Q1\_0, each weight keeps only its sign, with 0 for negative and 1 for positive. One FP16 scale is shared by every 128 weights. That works out to 1.125 effective bits per weight. The design is Qwen3-1.7B with 28 layers, grouped-query attention and a context of 32,768 tokens. The trade-off is that you need runtime kernels for this format. Download the Q1\_0 GGUF file → Build the PrismML llama.cpp fork → Load the file on CUDA, Metal or CPU → Generate text with the suggested settings ### Good to know The card lists the size of the weights alone in three formats. Weight memory by format | Label | Value | | --- | --- | | FP16 | 3.44 GB | | MLX 1-bit g128 | 0.27 GB | | GGUF Q1\_0 | 0.24 GB | - File size: the file on disk is 0.25 GB, a little above the weights alone, since text-handling data and metadata are packed in with them. - Size: 1.7B parameters, about 1.4B excluding embeddings. - MLX: a separate 1-bit build exists for Apple Silicon. - License: apache-2.0 - Cost to run: no API key, account or paid service named. The card also links a Colab notebook for trying it in a browser. - Data it reaches: files you download from Hugging Face, and the prompts you type. - Not for: runtimes without Q1\_0 kernels, since the card points to the PrismML forks. ## README ````markdown # Bonsai-1.7B-GGUF Generates text with a 1.7B language model packed to 1 bit per weight, about 0.24 GB, for llama.cpp on CUDA, Metal and CPU. For developers who need a tiny local model. Made by prism-ml on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Bonsai-1.7B is a small text generation model where every weight, including embeddings and the output head, is stored in a single bit. It uses the Qwen3-1.7B design, packed in the GGUF Q1_0 format. The weights take about 0.24 GB, against 3.44 GB at FP16, and it runs in llama.cpp on CUDA, Metal and CPU. Who it suits: Developers who need a language model on a small device or with little memory, and who are happy to use the maker's fork of llama.cpp. ```table Industry | Who uses it | How they use it | Example Mobile apps | App developers | Run a small model on a phone or tablet | The card lists Swift for iPhone and iPad, and Android Edge computing | Embedded engineers | Fit a language model into tight memory | Load a 0.24 GB model on a small machine Hobby projects | Local AI tinkerers | Try 1-bit models on a laptop | Chat with the model through llama.cpp on a Mac ``` Anyone who needs the best answer quality can skip it, since a 1.7B model at 1 bit trades quality for size. Where to use it: - llama.cpp on CUDA GPUs, including RTX and datacenter cards. - llama.cpp on Metal, for Macs. - llama.cpp on CPU. - Swift on iPhone and iPad, and Android. - Windows, macOS or Linux: the download steps run on all three with Python. ```table Use case | Who | What they get back On-device chat | App developers | Text replies from a model of about 0.24 GB Low-memory servers | Engineers | A language model that loads in a fraction of the FP16 memory Speed tests | Tinkerers | The card reports up to 3.0x faster than FP16 on an RTX 4090 ``` How to set it up: What you need first: - Python with pip, to download the file. - The PrismML fork of llama.cpp, which includes the Q1_0 kernels the model needs. - A CUDA GPU, a Mac with Metal, or a CPU. 1. Install the Hugging Face hub package with pip. 2. Download this repository with the hf command. 3. Build the PrismML fork of llama.cpp for your hardware, as the card describes. 4. Run the GGUF file with that build. ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-1.7B-gguf ``` How to use it: Run the file with the fork's llama.cpp tools. A short system prompt works, and the card suggests the sampling settings below. ```text You are a helpful assistant ``` ```table Title: Suggested generation settings Parameter | Default | Suggested range Temperature | 0.5 | 0.5 to 0.7 Top-k | 20 | 20 to 40 Top-p | 0.9 | 0.85 to 0.95 Repetition penalty | 1.0 | none given ``` Example: You start the model in the fork's llama.cpp with the system prompt above and ask a short question. It returns a plain text answer, generated from a file under 0.3 GB. For newcomers and experts: Most language models store each number in 16 bits. This one stores each in 1 bit, which is why the file is so small. A good first step is to download the file and chat with it in the fork of llama.cpp. In Q1_0, each weight keeps only its sign, with 0 for negative and 1 for positive. One FP16 scale is shared by every 128 weights. That works out to 1.125 effective bits per weight. The design is Qwen3-1.7B with 28 layers, grouped-query attention and a context of 32,768 tokens. The trade-off is that you need runtime kernels for this format. ```flow Download the Q1_0 GGUF file Build the PrismML llama.cpp fork Load the file on CUDA, Metal or CPU Generate text with the suggested settings ``` Good to know: The card lists the size of the weights alone in three formats. ```chart Title: Weight memory by format FP16: 3.44 GB MLX 1-bit g128: 0.27 GB GGUF Q1_0: 0.24 GB ``` - File size: the file on disk is 0.25 GB, a little above the weights alone, since text-handling data and metadata are packed in with them. - Size: 1.7B parameters, about 1.4B excluding embeddings. - MLX: a separate 1-bit build exists for Apple Silicon. - License: apache-2.0 - Cost to run: no API key, account or paid service named. The card also links a Colab notebook for trying it in a browser. - Data it reaches: files you download from Hugging Face, and the prompts you type. - Not for: runtimes without Q1_0 kernels, since the card points to the PrismML forks. ## On Hugging Face, 13 November 2026 - Task: text generation - Library: llama.cpp - Parameters: 1.7B - Weights: about 860 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 173,963 - Likes: 100 - License: apache-2.0 - Based on: prism-ml/Bonsai-1.7B-unpacked - Quantized versions: 3 - Fine-tunes: 3 - Tags: llama.cpp, gguf, 1-bit, llama-cpp, cuda, metal, on-device, prismml, bonsai, text-generation, conversational ## Links - Model: https://huggingface.co/prism-ml/Bonsai-1.7B-gguf - Files: https://huggingface.co/prism-ml/Bonsai-1.7B-gguf/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download prism-ml/Bonsai-1.7B-gguf ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [prism-ml/Bonsai-1.7B-gguf on Hugging Face](https://huggingface.co/prism-ml/Bonsai-1.7B-gguf) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Bonsai-1.7B-GGUF (this page) | 173,963 | 100 | 1.7B | apache-2.0 | 2026-04-18 | | [Bonsai-27B-gguf](https://mindket.com/p/0f4c3b2e-9ef6-4680-896e-4319b270e9f8) | 179,818 | 945 | 26.9B | apache-2.0 | 2026-07-17 | | [1-bit Bonsai 27B (MLX)](https://mindket.com/p/d7dcb13a-512c-4d6e-8d42-d76b91e7140d) | 1,003,554 | 291 | 1.7B | apache-2.0 | 2026-07-14 | | [Bonsai-1.7B-unpacked](https://mindket.com/p/14bbc17d-6c93-4d4f-a6a2-1bb2c0327eb6) | 2,362 | 15 | 1.7B | apache-2.0 | 2026-04-16 | | [MNN-Pocket-0.3B-GGUF](https://mindket.com/p/1e60bd47-21da-4333-9781-f0f82b9a2ebb) | 12,467 | 20 | 362M | apache-2.0 | 2026-11-10 | | [MiniCPM5-2B-GGUF](https://mindket.com/p/47f50d35-3645-40b1-914d-63bedd57f448) | 481,119 | 393 | 2.5B | apache-2.0 | 2026-11-11 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-04-18 - First published: 2026-03-29 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open prism-ml/Bonsai-1.7B-gguf on Hugging Face](https://huggingface.co/prism-ml/Bonsai-1.7B-gguf) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/976ed9c8-c475-4a88-a05b-e149421d5c28) - [Its page on Mindket](https://mindket.com/p/976ed9c8-c475-4a88-a05b-e149421d5c28)