OpenBonsai Qwen3.5-9B Ternary

OpenBonsai Qwen3.5-9B Ternary

Runs Qwen3.5-9B squeezed to ternary weights in a roughly 2 GB GGUF file, for CPU use in two llama.cpp variants. A research preview for people who experiment with extreme compression.

Made by webbrain-one on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: OpenBonsai Qwen3.5-9B Ternary is Qwen3.5-9B shrunk to ternary weights by webbrain-one, using the open OpenBonsai compression framework. Each weight takes one of three values, and the whole model is one GGUF file of about 2 GB. It handles text only, and this is the second release of a research preview.

Who it suits: It suits researchers and tinkerers who want to see how a 9B model behaves with ternary weights, and developers who want a small model on a CPU-only machine.

Industry | Who uses it | How they use it | Example
Research | Compression researchers | Test how a 9B model behaves with ternary weights | Compare it with the first release
Edge computing | Developers on CPU-only machines | Try a 9B model in about 2 GB of RAM | Run it with llama-server on a laptop
Education | Students of model compression | Study a model squeezed to three weight values | Read how the weights are grouped

Anyone who needs a finished product, or a model that reads images, can skip it, since this is a text-only research preview.

Where to use it:

• Prism's llama.cpp fork, one of the two runtimes the card tested
• ik_llama.cpp, the other tested runtime, which added support for this format
• A CPU, since the card tested CPU builds and calls the CUDA builds untested
• Windows, macOS or Linux, where the files download with Python

Use case | Who | What they get back
Local text chat | Tinkerers | Written replies from a tiny file on a CPU
Compression research | Researchers | A ternary model to test against the original
Code experiments | Developers | A small model whose coding scores the card reports

How to set it up:

• Python with pip, for the download
• A build of Prism's llama.cpp fork or of ik_llama.cpp, because upstream llama.cpp does not support this format
• About 2 GB of free RAM for the model

1. Install the Hugging Face hub client.
2. Download the model's files.
3. Build one of the two runtimes from its source code.
4. Run the model file with that runtime.

pip install -U huggingface_hub
hf download webbrain-one/openbonsai-qwen-3.5-9b

How to use it: Run llama-cli with the model file, a prompt and a limit on new tokens, or start llama-server for a local server. The chat template is built into the file. Qwen3.5's template turns thinking on by default, and the card says this model then often thinks past 4,096 tokens.

llama-cli -m OpenBonsai-Qwen3.5-9B-PTQ1_0.gguf -p "Explain why the sky is blue." -n 256

llama-server -m OpenBonsai-Qwen3.5-9B-PTQ1_0.gguf -rtr 1 -t 4 --port 8080

Option | What it does
-m | Picks the model file
-p | Gives the prompt text
-t 4 | Uses 4 CPU threads
-rtr 1 | Repacks the ternary weights while loading, for ik_llama.cpp's fast CPU kernels

Example: You run llama-cli with the card's own test prompt, Explain why the sky is blue, and read the written answer. One user in the card reports roughly 10 tokens per second, using 4 threads on a laptop with an i5-13420H processor.

For newcomers and experts: Ternary means each weight is only minus one, zero or plus one, which is why the file is so small. Treat it as an experiment and compare its answers with a normal model before relying on it.

Weights sit in groups of 128, each with one fp16 scale, and all 202 large matrices are ternary, embedding and output head included. The layout matches Prism ML's Ternary-Bonsai-2-9B. The file has no vision tower and no MTP layers.

Start from Qwen3.5-9B
OpenBonsai compresses it to ternary weights
One GGUF file of about 2 GB
A llama.cpp variant loads it
Text goes in and text comes out

Title: HumanEval problems passed with thinking off
Subset of 50, second release: 34
Subset of 50, first release: 19
Other 114, second release: 62
Other 114, first release: 45
Other 114 minus flagged 34, second release: 38
Other 114 minus flagged 34, first release: 26

Good to know:

• Status: a research preview from an independent project with no tie to Prism ML
• Results: the card says its math and code still trail the original model, and warns that the 34-of-50 score likely flatters the model on unseen problems
• Size: the file is about 2 GB, while the record's rough 4-bit estimate says about 4.5 GB
• Runtime: only Prism's fork and ik_llama.cpp load it, and CUDA builds are untested
• Versions: both releases use the same file name, and a checksum is the only way to distinguish them
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the prompts you give it on your own computer, while the download and build steps fetch files from the web
• Not for: finished products or image input, since it is a text-only research preview

On Hugging Face, 13 November 2026

• Library: gguf
• Parameters: 9B
• Weights: about 4.5 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 929
• Likes: 9
• License: apache-2.0
• Based on: Qwen/Qwen3.5-9B
• Tags: gguf, ternary, 1.58-bit, ptq1_0, qwen3.5, openbonsai, research-preview, conversational

Links

• Model: https://huggingface.co/webbrain-one/openbonsai-qwen-3.5-9b
• Files: https://huggingface.co/webbrain-one/openbonsai-qwen-3.5-9b/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download webbrain-one/openbonsai-qwen-3.5-9b

Then follow the usage in its model card. Nothing in this kit runs by itself.