Groot-19B-A3B-TTQ-GGUF

Groot-19B-A3B-TTQ-GGUF

Offers eleven GGUF sizes of the Groot-19B-A3B reasoning model for llama.cpp, from 7.88 to 18.59 GiB, plus a vision projector. Made for people who run the model on their own computer.

Made by TokenTinkerer on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds eleven GGUF versions of Groot-19B-A3B, an open-weight reasoning model from GrootTeam. They range from 3.56 to 8.50 bits per weight, so you can trade file size against fidelity. The repo also ships a separate Q8_0 projector file, which lets the model take images.

Who it suits: People who want to run Groot locally with llama.cpp and need to match the file to their memory. Quantization researchers can also use the measured numbers. The card says the model is not meant for general knowledge.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a reasoning model locally for tool-using workloads | Load one rung in llama.cpp for a coding agent
Self-hosting | Small teams | Run an open-weight model without a hosted API | Pick a mid-size file that fits the team's memory
Research | Quantization researchers | Compare file size against drift from the BF16 reference | Read the KLD columns for each rung
Hobby | Local model fans | Try several sizes of one model on a home computer | Start with a smaller file and move up

People who need a broad general-knowledge model can skip it, because the author says the compact design costs some knowledge.

Where to use it:

• llama.cpp, the library this model is tagged for
• Windows, macOS or Linux, with Python for the download tools
• A setup that supports images, using the projector file next to one of the model files

Use case | Who | What they get back
Local text generation | Developers | Replies from a reasoning model running on your own machine
Image input | Users of vision-capable setups | A way to feed pictures to the model through the projector
Quality comparison | Researchers | Measured drift and top-token agreement for each file size

How to set it up:

• Python, for the Hugging Face hub tools
• llama.cpp, to load and run a GGUF file
• Disk space for one file, from 7.88 GiB to 18.59 GiB, plus 586 MiB if you want the projector

1. Install the Hugging Face hub tools with pip.
2. Download the files with the hf command. The repo holds every rung, so a full download is large.
3. Pick the one file that fits your memory and keep it.
4. Load that file in llama.cpp.

pip install -U huggingface_hub
hf download TokenTinkerer/Groot-19B-A3B-TTQ-GGUF

How to use it: Choose a file by size. Smaller files drift further from the BF16 reference, and larger ones stay closer. File names follow one pattern, with the bits per weight and the type in the name. The card has no chat command in the part shown, so run the chosen file the way llama.cpp normally loads a GGUF.

Type | Size | Use
IQ3_XXS | 7.88 GiB | Smallest rung, with the largest measured drift
Q3_K_M | 9.06 GiB | A small file with moderate drift
IQ4_XS | 9.60 GiB | Smaller than the upstream Q4_K_M, with close to the same drift on code
Q4_K_M | 11.17 GiB | A little larger than the upstream file, with lower drift
Q5_K_M | 13.63 GiB | More room in memory for closer output
Q6_K | 14.96 GiB | Close to the reference
Q8_0 | 18.59 GiB | Largest rung, nearest to the BF16 reference

Example: You have limited memory and pick the IQ4_XS file at 9.60 GiB instead of the upstream Q4_K_M at 10.86 GiB. By the card's measurements the file is 11.6% smaller, and its drift on code is close to the same.

For newcomers and experts: A GGUF file is one packed copy of a model that llama.cpp can load. Quantization shrinks it by storing weights with fewer bits. For a first try, take the Q4_K_M file, since it sits near the middle of the range.

The author measures KL divergence against the BF16 model's logits over 40 fixed chunks of 512 tokens, on a code set and on wikitext-2 test. Top-1 agreement is reported next to it. All the rungs start from a BF16 copy of the original weights, with no second quantization step in between, so none carries older rounding error. The per-tensor recipe was reused from two other ladders because the models share the same 753 tensor names.

Convert the original model to BF16
Quantize it into eleven rungs
Measure KL divergence against the BF16 logits
Publish each file with its measured numbers

Good to know:

• Size: The eleven files total 124.75 GiB, and the BF16 reference file in the repo is 35.4 GiB.
• Focus: Upstream describes Groot as a family for reasoning and tool use. The card says it is not a general-knowledge model.
• Measured: The author says the figures were measured on one machine, not estimated.

Title: File size of selected rungs
IQ3_XXS: 7.88 GiB
Q3_K_M: 9.06 GiB
IQ4_XS: 9.60 GiB
Q4_K_M: 11.17 GiB
Q5_K_M: 13.63 GiB
Q6_K: 14.96 GiB
Q8_0: 18.59 GiB

• License: Apache 2.0.
• Cost to run: none named. No API key or paid service appears in the source.
• Data it reaches: The model files you download from Hugging Face and the prompts or images you give llama.cpp.
• Not for: General knowledge questions. The one knowledge benchmark the author ran shows where compactness cost accuracy.

On Hugging Face, 14 November 2026

• Task: text generation
• Library: llama.cpp
• Parameters: 19B
• Weights: about 9.5 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 6,541
• Likes: 4
• License: apache-2.0
• Languages: en, zh
• Based on: GrootTeam/Groot-19B-A3B-261023
• Tags: llama.cpp, gguf, quantized, imatrix, qwen35moe, vision, custom-quants, text-generation, conversational

Links

• Model: https://huggingface.co/TokenTinkerer/Groot-19B-A3B-TTQ-GGUF
• Files: https://huggingface.co/TokenTinkerer/Groot-19B-A3B-TTQ-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download TokenTinkerer/Groot-19B-A3B-TTQ-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.