--- title: "google_gemma-4-31B-it-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/13999cd3-07fe-4827-8118-4b90ee42ae50" updated: "2026-11-13" --- # google\_gemma-4-31B-it-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Quantized GGUF files of Google's Gemma 4 31B instruction model, from 17.15 GB to 61.41 GB, made by bartowski with an importance matrix. Reads images and text. Run in llama.cpp, LM Studio and similar tools. google\_gemma-4-31B-it-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by bartowski ## About ### What it is This repository holds GGUF quantizations of Google's gemma-4-31B-it, made by bartowski. He built them with llama.cpp release b8746 and an importance matrix, which helps keep quality as files shrink. The model has 30.7B parameters and takes images and text. There are many file sizes, from a 61.41 GB full BF16 copy down to files near 17 GB. The card says its latest update brought fixes and a chat template. ### Who it suits It suits people who want to run Gemma 4 31B on their own computer and need a file size that fits their memory. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI | People running models offline | Load a quantized file in a desktop runner | Run the 19.60 GB Q4\_K\_M file in LM Studio | | Software | Developers testing local models | Serve the model from llama.cpp or a similar tool | Try several quant levels on the same prompts | | Research | Hobbyists who compare quantization levels | Test two file sizes on the same questions | Compare Q6\_K with Q5\_K\_M on one set of prompts | You can skip it if you want the original Google weights for training, since these are compressed files for running the model. ### Where to use it - llama.cpp, ramalama, koboldcpp or LM Studio - Jan AI, Text Generation Web UI or LoLLMs - Windows, macOS and Linux with Python, to download the files | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Hobbyists | Replies from a 31B model on their own machine | | Image questions | Users of vision capable runners | Answers that take a picture into account | | Size choice | People with limited memory | A file that fits the RAM or VRAM they have | ### How to set it up You need these first: - Python, to use Hugging Face's download client - Disk space and memory that fit the file you pick - One of the runners listed above 1. Install or update the Hugging Face download client. 2. Choose one file from the card's table, since the card says not to fetch the whole branch. 3. Download that file to your computer. 4. Load it in your runner and use the prompt format below. ```sh pip install -U huggingface_hub hf download bartowski/google_gemma-4-31B-it-GGUF ``` ### How to use it You load one file in your runner and chat with the model. If your runner does not apply the chat template itself, the card gives the prompt format below. The text available here does not show run commands. ```text <|turn>system {system_prompt} <|turn>user {prompt} <|turn>model <|channel>thought ``` | Tool | What it does | | --- | --- | | Q4\_K\_M | A 19.60 GB file that the card names as the default size for most uses | | Q6\_K | A 26.73 GB file with higher quality | | IQ4\_XS | A 17.16 GB file that saves space compared with the Q4 files | | Q3\_K\_XL | A 17.15 GB file with lower quality, for low memory | | bf16 | The full weights at 61.41 GB, stored in several parts | ### Example You have a machine with about 24 GB of memory, so you take the 19.60 GB Q4\_K\_M file. You load it in LM Studio and send a prompt. The card suggests this size for most people. ### For newcomers and experts Quantizing shrinks a model so it fits in less memory, and the suffix tells how much was shrunk. A good first try is the Q4\_K\_M file, then a larger or smaller one if it does not fit. For readers who know the field, the files marked L keep the embedding and output weights at Q8\_0. Q4\_0 and IQ4\_NL offer online repacking for some ARM and AVX CPUs. Q4\_1 is a legacy format that the card says improves tokens per watt on Apple silicon. Pick a file that fits your memory → Download that single file → Open it in a runner → Apply the prompt format → Chat with the model ### Good to know - Single file: the card says to download one file, not the whole branch, and the repository holds many large files. - New models: tools may need an update before a newly supported model runs. - Cut: the file table in the text available here stops after 15 rows, so the smaller files are missing. - Counts: Hugging Face lists 30.7B parameters and about 15.3 GB at 4-bit as an estimate. - Versions: Hugging Face lists no other quantized versions and 1 fine tuned version. File size by quantization, from the card's table | Label | Value | | --- | --- | | bf16 | 61.41 GB | | Q8\_0 | 32.64 GB | | Q6\_K | 26.73 GB | | Q5\_K\_M | 22.61 GB | | Q4\_K\_M | 19.60 GB | | IQ4\_XS | 17.16 GB | | Q3\_K\_XL | 17.15 GB | - License: Apache 2.0. - Cost to run: none named. You need memory that fits the file. - Data it reaches: the prompts and images you give it, on your own machine. The text available here names no network use. - Not for: none named in the text available here. ## README ````markdown # google_gemma-4-31B-it-GGUF Quantized GGUF files of Google's Gemma 4 31B instruction model, from 17.15 GB to 61.41 GB, made by bartowski with an importance matrix. Reads images and text. Run in llama.cpp, LM Studio and similar tools. Made by bartowski on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF quantizations of Google's gemma-4-31B-it, made by bartowski. He built them with llama.cpp release b8746 and an importance matrix, which helps keep quality as files shrink. The model has 30.7B parameters and takes images and text. There are many file sizes, from a 61.41 GB full BF16 copy down to files near 17 GB. The card says its latest update brought fixes and a chat template. Who it suits: It suits people who want to run Gemma 4 31B on their own computer and need a file size that fits their memory. ```table Industry | Who uses it | How they use it | Example Local AI | People running models offline | Load a quantized file in a desktop runner | Run the 19.60 GB Q4_K_M file in LM Studio Software | Developers testing local models | Serve the model from llama.cpp or a similar tool | Try several quant levels on the same prompts Research | Hobbyists who compare quantization levels | Test two file sizes on the same questions | Compare Q6_K with Q5_K_M on one set of prompts ``` You can skip it if you want the original Google weights for training, since these are compressed files for running the model. Where to use it: - llama.cpp, ramalama, koboldcpp or LM Studio - Jan AI, Text Generation Web UI or LoLLMs - Windows, macOS and Linux with Python, to download the files ```table Use case | Who | What they get back Local chat | Hobbyists | Replies from a 31B model on their own machine Image questions | Users of vision capable runners | Answers that take a picture into account Size choice | People with limited memory | A file that fits the RAM or VRAM they have ``` How to set it up: You need these first: - Python, to use Hugging Face's download client - Disk space and memory that fit the file you pick - One of the runners listed above 1. Install or update the Hugging Face download client. 2. Choose one file from the card's table, since the card says not to fetch the whole branch. 3. Download that file to your computer. 4. Load it in your runner and use the prompt format below. ```sh pip install -U huggingface_hub hf download bartowski/google_gemma-4-31B-it-GGUF ``` How to use it: You load one file in your runner and chat with the model. If your runner does not apply the chat template itself, the card gives the prompt format below. The text available here does not show run commands. ```text <|turn>system {system_prompt} <|turn>user {prompt} <|turn>model <|channel>thought ``` ```table Tool | What it does Q4_K_M | A 19.60 GB file that the card names as the default size for most uses Q6_K | A 26.73 GB file with higher quality IQ4_XS | A 17.16 GB file that saves space compared with the Q4 files Q3_K_XL | A 17.15 GB file with lower quality, for low memory bf16 | The full weights at 61.41 GB, stored in several parts ``` Example: You have a machine with about 24 GB of memory, so you take the 19.60 GB Q4_K_M file. You load it in LM Studio and send a prompt. The card suggests this size for most people. For newcomers and experts: Quantizing shrinks a model so it fits in less memory, and the suffix tells how much was shrunk. A good first try is the Q4_K_M file, then a larger or smaller one if it does not fit. For readers who know the field, the files marked L keep the embedding and output weights at Q8_0. Q4_0 and IQ4_NL offer online repacking for some ARM and AVX CPUs. Q4_1 is a legacy format that the card says improves tokens per watt on Apple silicon. ```flow Pick a file that fits your memory Download that single file Open it in a runner Apply the prompt format Chat with the model ``` Good to know: - Single file: the card says to download one file, not the whole branch, and the repository holds many large files. - New models: tools may need an update before a newly supported model runs. - Cut: the file table in the text available here stops after 15 rows, so the smaller files are missing. - Counts: Hugging Face lists 30.7B parameters and about 15.3 GB at 4-bit as an estimate. - Versions: Hugging Face lists no other quantized versions and 1 fine tuned version. ```chart Title: File size by quantization, from the card's table bf16: 61.41 GB Q8_0: 32.64 GB Q6_K: 26.73 GB Q5_K_M: 22.61 GB Q4_K_M: 19.60 GB IQ4_XS: 17.16 GB Q3_K_XL: 17.15 GB ``` - License: Apache 2.0. - Cost to run: none named. You need memory that fits the file. - Data it reaches: the prompts and images you give it, on your own machine. The text available here names no network use. - Not for: none named in the text available here. ## On Hugging Face, 13 November 2026 - Task: image text to text - Parameters: 30.7B - Weights: about 15.3 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 44,117 - Likes: 98 - License: apache-2.0 - Based on: google/gemma-4-31B-it - Fine-tunes: 1 - Tags: gguf, image-text-to-text, imatrix, conversational ## Links - Model: https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF - Files: https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download bartowski/google_gemma-4-31B-it-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [bartowski/google_gemma-4-31B-it-GGUF on Hugging Face](https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | google\_gemma-4-31B-it-GGUF (this page) | 44,117 | 98 | 30.7B | apache-2.0 | 2026-07-27 | | [Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | | [Qwen3.6-35B-A3B-MTP-GGUF](https://mindket.com/p/90db7fe0-7044-4238-a508-5d2ecb760999) | 2,315,167 | 1,051 | 35.5B | apache-2.0 | 2026-05-20 | | [Qwen3.6-35B-A3B-GGUF](https://mindket.com/p/9ac02ca9-6ba3-4ea0-b3f6-dcaea6e7789f) | 1,649,871 | 1,789 | 34.7B | apache-2.0 | 2026-04-20 | | [Qwen3.8-27B-GSQ-RCO-GGUF](https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40) | 1,358,588 | 2,739 | 26.9B | apache-2.0 | 2026-09-02 | | [DeepSeek-V4.1-Flash-1017-GGUF](https://mindket.com/p/1c185af8-98d2-4feb-a1a3-d5821c72863f) | 1,295,742 | 385 | 748B | mit | 2026-11-02 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-07-27 - First published: 2026-04-02 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open bartowski/google_gemma-4-31B-it-GGUF on Hugging Face](https://huggingface.co/bartowski/google_gemma-4-31B-it-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/13999cd3-07fe-4827-8118-4b90ee42ae50) - [Its page on Mindket](https://mindket.com/p/13999cd3-07fe-4827-8118-4b90ee42ae50)