gemma-4-26B-A4B-it Cactus CQ

gemma-4-26B-A4B-it Cactus CQ

2-bit, 3-bit and 4-bit compressed copies of Google's gemma-4-26B-A4B-it chat model in the Cactus weights format. Built to run on ARM devices through the Cactus runtime.

Made by Cactus-Compute on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds three compressed versions of google/gemma-4-26B-A4B-it, made by Cactus-Compute. They use 2, 3 and 4 bits per weight and are stored in the Cactus weights format for running on ARM devices.

The 2-bit and 3-bit versions are calibrated, so they keep quality close to the original at a much smaller size. The 4-bit version uses plain round-to-nearest rounding without calibration.

Who it suits: Developers who build apps that run a large chat model directly on an ARM device and use the Cactus runtime. The model is a mixture-of-experts design, and the files are shaped for that.

Industry | Who uses it | How they use it | Example
Mobile apps | App developers | Run a chat model on the device instead of a server | An ARM phone app that answers without a network
Embedded systems | Hardware engineers | Fit a large model into limited memory | Choosing the 2-bit file for a small device
Research | Quantization researchers | Study calibrated low-bit methods on an expert model | Comparing perplexity across bit widths
Privacy-minded teams | Product teams | Keep prompts on the device | An assistant that never sends text out

Anyone who needs x86 desktop inference or a plain Transformers checkpoint can skip it, since the card covers only the Cactus ARM runtime.

Where to use it:

• ARM devices, through the Cactus runtime.
• Windows, macOS or Linux with Python, for downloading the files.

Use case | Who | What they get back
On-device chat | App developers | Text replies produced locally on ARM hardware
Size tuning | Engineers | A choice of 2, 3 or 4 bits per weight
Quality comparison | Researchers | Published perplexity for each variant

How to set it up: You need the Cactus runtime for ARM, which this page does not describe, and Python for the download.

• Python on the computer you download with
• The huggingface_hub package
• An ARM device with the Cactus runtime
• Acceptance of the Gemma license terms

1. Install the Hugging Face tool.
2. Download the repository, or just the file for the bit width you want.
3. Unzip it and load it in the Cactus runtime on your device.

pip install -U huggingface_hub
hf download Cactus-Compute/gemma-4-26B-A4B-it

How to use it: Pick one of the three zip files, load it in the Cactus runtime, and chat with it as you would with the original model. The card gives no commands for the runtime itself.

The three files hold the 2-bit calibrated version at about 2.36 bits per weight overall, the 3-bit calibrated version and the 4-bit round-to-nearest version.

Example: A developer building an ARM app downloads the 3-bit file, loads it in the Cactus runtime and asks a question in chat. The reply is generated on the device. What speed to expect is not stated in the card.

Title: Held-out perplexity, lower is better
bf16 baseline: 7.25 PPL
2-bit calibrated: 6.81 PPL
3-bit calibrated: 6.32 PPL
4-bit RTN: 6.00 PPL

For newcomers and experts: If you are new, quantization means storing a model's numbers with fewer bits so the model fits in less memory. The card measured how well each version predicts held-out answers, and lower perplexity means better. Start with the 4-bit file if your device has room.

For experts, the method is codebook quantization with a per-group Hadamard rotation. The 2-bit and 3-bit files add activation scaling in the style of AWQ and a GPTQ step that is aware of routing, so each expert is calibrated on the tokens routed to it. Embeddings use 4-bit codebooks, while norms, router and biases stay in FP16. Calibration used about 2M tokens from WildChat and AceCode, generated by the model with thinking on. The 4-bit version skips calibration because calibrating there made results worse. Uncalibrated 2-bit and 3-bit rounding gave perplexity of 33,827 and 32.56, and the card notes about two points of noise.

Start from google/gemma-4-26B-A4B-it
Calibrate each expert on its own routed tokens
Quantize to 2, 3 or 4 bits
Package the weights as Cactus zip files
Run them on ARM with the Cactus runtime

Good to know: The points below cover limits and costs.

• Format: the files only work with the Cactus runtime on ARM.
• License: Gemma, the license listed for the base model.
• Cost to run: no API key or paid service is named.
• Data it reaches: nothing beyond the prompts you send to it. Downloading the files needs a connection to Hugging Face.
• Not for: runtimes other than Cactus, or devices that are not ARM.

On Hugging Face, 14 November 2026

• Task: text generation
• Downloads in the last 30 days: 41
• Likes: 6
• License: gemma
• Based on: google/gemma-4-26B-A4B-it
• Tags: gemma4, cactus, quantized, on-device, text-generation

Links

• Model: https://huggingface.co/Cactus-Compute/gemma-4-26B-A4B-it
• Files: https://huggingface.co/Cactus-Compute/gemma-4-26B-A4B-it/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Cactus-Compute/gemma-4-26B-A4B-it

Then follow the usage in its model card. Nothing in this kit runs by itself.