gemma-4-12B-it GGUF

gemma-4-12B-it GGUF

Runs Google's gemma-4-12B-it on your own computer as a GGUF file that ggml-org converted automatically, through the Llama app or the llama command. It suits people who run models locally.

Made by ggml-org on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds a GGUF copy of Google's gemma-4-12B-it, converted automatically by ggml-org. The record lists 11.9B parameters and the task any to any.

The card names four Google source models, including an assistant version and two QAT q4_0 unquantized versions. It does not say which quantization types the files use.

Who it suits: People who run language models on their own computers suit it best, including developers who want a local Gemma to test.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists | Run a 12B model on their own machine | Start the model from a terminal
Software | Developers | Serve a local model to their own apps | Run the llama serve command
Research | Model testers | Try a Google model in GGUF form | Compare it with other local models

Anyone who needs details on the quantization can skip it, because the card is brief and its own to-do list says it needs more information.

Where to use it: You run it on your own computer, in an app or from a terminal.

• The Llama app
• A terminal, with the llama command
• Windows, macOS or Linux

Use case | Who | What they get back
Local chat | Hobbyists | A model that answers on your own machine
Local serving | Developers | A running model your apps can use
Model testing | Testers | A Gemma model in GGUF form to compare

How to set it up: You need the Llama app or the llama command, and optionally Python to download the files yourself.

• The Llama app or the llama command-line tool
• Optional: Python with the Hugging Face hub tools

1. Install the Llama app or the llama command-line tool.
2. Optionally download the model files with the hf command.
3. Run the serve command from a terminal.

pip install -U huggingface_hub
hf download ggml-org/gemma-4-12B-it-GGUF

How to use it: Run the command below in a terminal. The hf option names the Hugging Face repository that the command loads and serves.

llama serve -hf ggml-org/gemma-4-12B-it-GGUF

Tool | What it does
Llama app | Runs the model in an app
llama serve | Loads the repository you name after the hf option and serves it

Example: You open a terminal and run llama serve with the repository name. The tool loads this model and serves it on your computer, ready for the app or program you point at it.

For newcomers and experts: GGUF is a file format for running a model locally. Your first small try is to open the Llama app and load this model, or to run the command above.

Under the hood, an automatic converter from ggml-org produced the files from Google's published models. The record estimates about 6 GB at 4-bit for the weights, but that is an estimate, and the card gives no list of the quantization types.

Google's gemma-4-12B-it
Automatic conversion by ggml-org
GGUF files in this repository
Served locally by llama

Good to know: The card is short and unfinished, so treat details beyond the command as unconfirmed.

• Size: 11.9B parameters, with an estimate of about 6 GB at 4-bit
• Other versions: Hugging Face lists 0 quantized and 1 fine-tuned version made by other people
• License: apache-2.0
• Cost to run: none named
• Data it reaches: the prompts you give it, and the model files downloaded from Hugging Face
• Status: the card's to-do list says information is still to be added

On Hugging Face, 13 November 2026

• Task: any to any
• Parameters: 11.9B
• Weights: about 6 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 107,736
• Likes: 106
• License: apache-2.0
• Based on: google/gemma-4-12B-it
• Fine-tunes: 1
• Tags: gguf, quantized, any-to-any, conversational

Links

• Model: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF
• Files: https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ggml-org/gemma-4-12B-it-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.