ZONOS3-Mini GGUF

ZONOS3-Mini GGUF

Packs the ZONOS3-Mini text-to-speech model as GGUF files for llama.cpp, with 27 built-in voices and six backbone sizes from 0.41 to 1.61 GB. For developers running speech locally.

Made by AtomicChat on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF files of Zyphra's ZONOS3-Mini text-to-speech model, made by the AtomicChat team for llama.cpp. An 800M-parameter backbone produces the speech codes, and a codec named ZAC converts them to audio at 44.1 kHz. Ten languages are covered, with 27 voices built in.

Who it suits: It suits developers who want to run text-to-speech locally with llama.cpp, and people comparing quantization levels for a speech model.

Title: Who might use it
Industry | Who uses it | How they use it | Example
App development | Developers | Add offline speech to a local app | A desktop assistant that reads replies aloud
Local AI tools | Hobbyists | Run speech next to other llama.cpp models | A home setup that speaks in several languages
Speech research | Researchers | Compare file sizes against error rate and voice similarity | Weighing the 0.41 GB file against the full BF16 one

People who want a stable, merged llama.cpp build can skip it for now, since support was still an open pull request when the files were made.

Where to use it:

• llama.cpp, built at the pull request that adds ZONOS3 support
• NVIDIA GPUs, using the CUDA build option
• Apple Silicon Macs, where Metal is on by default
• Windows, macOS or Linux: download the files with Python

Title: Jobs it does
Use case | Who | What they get back
Local speech synthesis | Developers | 44.1 kHz audio from text, using one of the built-in voices
Size trade-offs | Researchers | Six backbone files from 0.41 to 1.61 GB to compare
Multilingual voices | App teams | A model the card tests in English, German and Chinese

How to set it up: You need these first:

• Python on Windows, macOS or Linux, to download the files
• A llama.cpp build made from the ZONOS3 pull request, with the CUDA option for NVIDIA
• Two files to load together: a backbone and an mmproj, the latter containing the voices, the codebook heads and the codec decoder

1. Install the Hugging Face hub package.
2. Download the repository files.
3. Build llama.cpp at the pull request the card names.
4. Choose one backbone and one mmproj to load.

pip install -U huggingface_hub
hf download AtomicChat/ZONOS3-Mini-GGUF

The CUDA option for NVIDIA GPUs is this build flag.

-DGGML_CUDA=ON

How to use it: Load the backbone and the mmproj together in your llama.cpp build and give it text to speak. The card's Run it section holds the exact commands. This command fetches the files.

hf download AtomicChat/ZONOS3-Mini-GGUF

The card recommends AD-Q4_K_M with the F16 mmproj, about 1.13 GB in total. It reports this pair as within noise of BF16 except for a small drop in German voice similarity. AD-Q3_K_M is the smallest, and its losses are small but measurable.

Example: You load AD-Q4_K_M with the F16 mmproj and ask for an English sentence in the British female voice Imogen, like the card's samples. Audio comes back at 44.1 kHz.

For newcomers and experts: If you are new to this, start with the recommended pair, about 1.13 GB, once your llama.cpp build is ready. Try one short sentence in English before other languages.

If you work in the field, the backbone is a mixture-of-experts model with about 100M parameters active per frame, mixing Gated DeltaNet and attention, and it predicts 10 codebooks per frame. In the AD files every part outside the experts stays at Q8_0, and the experts' ffn_down tensors get one level more precision. The stock Q4_K_M preset is similar in size but loses more on Chinese, by 1.8 points of character error rate.

Text goes in with a chosen voice
The backbone predicts 10 codebooks per frame
The mmproj holds the voices and codec decoder
The ZAC codec writes 44.1 kHz audio

Good to know: The card lists six backbone files, and the mmproj comes in two sizes, 1.07 GB for F32 and 0.62 GB for F16.

Title: Backbone file size in GB
BF16: 1.61 GB
Q8_0: 0.85 GB
AD-Q6_K: 0.67 GB
AD-Q5_K_M: 0.61 GB
AD-Q4_K_M: 0.51 GB
AD-Q3_K_M: 0.41 GB

• License: apache-2.0
• Cost to run: none named
• Data it reaches: only the text you give it, run locally with llama.cpp
• Not for: stock llama.cpp until the pull request merges, or Zyphra's own ggml runtime, whose files do not load here

On Hugging Face, 13 November 2026

• Task: text to speech
• Library: gguf
• Parameters: 802M
• Weights: about 401 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 385
• Likes: 9
• License: apache-2.0
• Languages: en, de, zh, fr, es, it, pt, ko
• Based on: Zyphra/ZONOS3-Mini
• Tags: gguf, quantized, llama.cpp, text-to-speech, zonos3

Links

• Model: https://huggingface.co/AtomicChat/ZONOS3-Mini-GGUF
• Files: https://huggingface.co/AtomicChat/ZONOS3-Mini-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download AtomicChat/ZONOS3-Mini-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.