gemma-4-12b-it-GGUF

gemma-4-12b-it-GGUF

Unsloth's GGUF quantized copy of Google's instruction-tuned Gemma 4 12B, a model that takes text, images, audio and video and writes text, for running locally.

Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: gemma-4-12b-it-GGUF is a GGUF copy of google/gemma-4-12B-it, made by Unsloth. The base is the Gemma 4 12B Unified model from Google DeepMind. The record lists 11.9B parameters, and the 4-bit weights come to about 6 GB by its estimate.

The model takes text, images, audio and video as input and writes text. The card says it understands audio and vision directly, without separate encoders.

Who it suits: It suits people who want a multimodal model on their own computer and have a machine that can hold a file of about 6 GB. Developers can also fine-tune it, as the card mentions Unsloth Studio for that.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local model for text, image and audio input | Asking what a screenshot shows
Agents | Tool builders | Use function calling and thinking modes | A helper that picks a tool for a request
Media | Editors | Get text answers about clips and recordings | Summarizing a short recording
Research | Hobbyist researchers | Try a recent open model on their own hardware | Comparing it with smaller Gemma 4 sizes

Skip it if you want a hosted assistant and have no plan to run a model.

Where to use it:

• Local apps that load GGUF files
• Unsloth Studio, which the card says can run and fine-tune this model
• Windows, macOS or Linux, with Python for the download step

Use case | Who | What they get back
Chat and reasoning | Developers | Text answers, with configurable thinking modes
Image and audio questions | App builders | Text answers about a picture or a recording
Function calling | Agent builders | Calls to tools driven by the model

How to set it up: You need a few things in place first.

• Python and pip, for the download command
• A local app that can load GGUF files
• About 6 GB of free disk space for the 4-bit weights, by the record's estimate

1. Install the Hugging Face hub package with pip.
2. Download the repository with the hf command.
3. Load the GGUF file in your local app. Unsloth keeps a guide for running Gemma 4 12B.

pip install -U huggingface_hub
hf download unsloth/gemma-4-12b-it-GGUF

How to use it: After loading the file, you chat with the model and may add images or audio. The card says Gemma 4 supports a system role for more controllable conversations and native function calling. It also lists multi-token prediction support through an Unsloth guide.

Example: You load the file and give the model a screenshot with the question "What does this error message mean?" The model reads the picture and answers in text.

For newcomers and experts: Start with one short text question. When that works, add an image. Audio and video come last, since they need more setup in the app you use.

The 12B Unified model reads audio and images without separate encoders. That makes the model encoder-free and keeps its size friendly to consumer hardware. Gemma 4 attention mixes local sliding windows with global layers, and the family reaches a context window of 256K tokens. The quantized copy trades some precision for a smaller file.

Download the GGUF file with hf
Load it in a local app
Send text, an image or audio
The model writes text back

Good to know:

• License: Apache 2.0
• Cost to run: none named, but you need a machine with room for about 6 GB of weights
• Data it reaches: it downloads files from Hugging Face onto your computer

On Hugging Face, 13 November 2026

• Task: image text to text
• Parameters: 11.9B
• Weights: about 6 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 2,534,530
• Likes: 963
• License: apache-2.0
• Based on: google/gemma-4-12B-it
• Quantized versions: 1
• Tags: gguf, gemma4, unsloth, gemma, google, gemma4_unified, image-text-to-text, conversational

Links

• Model: https://huggingface.co/unsloth/gemma-4-12b-it-GGUF
• Files: https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download unsloth/gemma-4-12b-it-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.