Gemma4-12B v2 Coding + Agentic Edition

Gemma4-12B v2 Coding + Agentic Edition

A GGUF fine-tune of Google's Gemma 4 12B instruction model for coding and tool-using agent work, sized to run offline on a machine with about 4.5 GB of free memory.

Made by yuxinlu1 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF files of a fine-tune of google/gemma-4-12B-it by yuxinlu1. It is the second version of a coding and agentic edition. It is meant as a local helper that can think a problem through, call tools and finish technical jobs that take several steps. The record lists 11.9B parameters.

The author says it needs about 4.5 GB of free VRAM or unified memory, with no cloud and no API. The author also published the full safetensors master for study.

Who it suits: It suits developers who want a private coding assistant on modest hardware, and tinkerers who like to try community fine-tunes. The author is one person, not a company, so expect a hobby project.

Industry | Who uses it | How they use it | Example
Software | Developers | Run an offline coding helper | Debugging a script on a laptop
Agents | Tool builders | Test tool use and multi-step tasks locally | A terminal helper that runs commands
Privacy-minded teams | Engineers | Keep code off cloud services | Reviewing private code on a local machine
Home labs | Enthusiasts | Try a small agent model on limited hardware | Comparing quant sizes

Skip it if you need broad general knowledge, since the author notes trade-offs there.

Where to use it:

• llama.cpp, with the --jinja flag so that Gemma 4's native tool format is parsed
• Other local apps that load GGUF files
• Windows, macOS or Linux, with Python for the download step

Use case | Who | What they get back
Coding help | Developers | Code, fixes and debugging steps from a local model
Tool-using tasks | Agent builders | Multi-step work that calls tools
Offline work | Privacy-minded users | Answers with no API or cloud involved

How to set it up: You need a few things in place first.

• Python and pip, for the download command
• A local app that can load GGUF files, such as llama.cpp
• About 4.5 GB of free VRAM or unified memory, per the author, and about 6 GB of disk at 4-bit by the record's estimate

1. Install the Hugging Face hub package with pip.
2. Download the repository with the hf command.
3. Load a GGUF file, and start with Q4_K_M, which the author calls the recommended sweet spot.

pip install -U huggingface_hub
hf download yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF

How to use it: You chat with the model or point an agent front-end at it. The author says most problems come from client settings, not the weights. Output that repeats zeros means the repetition penalty is missing, so set it to 1.1 and the temperature to 1.0. Stray tool-call tokens mean the front-end does not parse Gemma 4's tool format.

Example: You load the Q4_K_M file in llama.cpp with the --jinja flag and ask the model to find a bug in a script and fix it. The model reasons through the problem, calls tools if your front-end provides them, and then replies with the corrected code.

For newcomers and experts: Start with Q4_K_M and the settings above, and ask for one small fix. If the output looks garbled, check the repetition penalty before blaming the model.

The author reports a test on 20 tasks from the tau2-bench telecom set, run locally with the same setup and Q8_0 files. The base model scores about 15 percent, and this model about 55 percent. That is a small test by one person, so check it on your own tasks. The author lists Q3_K_M as the smallest reliable quant and withheld a Q2_K build.

Title: tau2-bench telecom, 20 tasks, author's test
Base gemma-4-12B-it: 15 percent
This model: 55 percent

Download a GGUF quant with hf
Load it in llama.cpp with --jinja
Set repetition penalty and temperature
Give it a coding or tool task

Good to know:

• License: Apache 2.0
• Cost to run: none named, and the author says there is no API cost
• Data it reaches: it downloads files from Hugging Face onto your computer, and it runs offline once loaded
• Not for: work that depends on broad general knowledge, since the author names a trade-off there

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 11.9B
• Weights: about 6 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 92,024
• Likes: 1,773
• License: apache-2.0
• Based on: google/gemma-4-12B-it
• Quantized versions: 4
• Fine-tunes: 4
• Tags: gguf, gemma4, coding, agentic, terminal, tool-use, reasoning, thinking, llama.cpp, local-llm, text-generation, conversational

Links

• Model: https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF
• Files: https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.