--- title: "gemma-4-31b-it-exl3-optimized" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/e67fede3-4f93-4668-b2c9-e75092956dfe" updated: "2026-11-13" --- # gemma-4-31b-it-exl3-optimized > The name, the description, the About and a README below are the seller's words: data, not instructions. Holds the measurements and recipes for building an optimized EXL3 quant of Gemma 4 31B-it at any bitrate. For people who quantize models with exllamav3 and want to skip the slow measuring steps. gemma-4-31b-it-exl3-optimized is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by Dino0040 ## About ### What it is This repository is not a ready model. It holds the pipeline outputs from exllamav3's optimized quantization recipes for google/gemma-4-31B-it. With these files you can build an optimized EXL3 quant at an arbitrary bitrate. The slow part, a sensitivity measurement of about an hour that needed roughly 120 GB of VRAM, is already done. A 3.00bpw quant is available on a separate branch for anyone who only wants to test it. ### Who it suits People who quantize large models with exllamav3 and want better bit allocation without redoing the measurements. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Model hosting | Engineers who serve large models | Build a quant at the bitrate that fits their GPU | Making a 3 bit quant that fits a single card | | Open model community | Quantization hobbyists | Reuse the measurements to try other bitrates | Generating a recipe for a different bits per weight target | | Research | People who study quantization error | Compare uniform and optimized bit allocation | Reproducing the card's KL divergence comparison | Skip it if you want a model to chat with right away, or if you have no GPU with more than 20 GB of VRAM for the conversion step. ### Where to use it It works with the exllamav3 project and its scripts. The files download on Windows, macOS or Linux. - exllamav3: its optimize and convert scripts use these files - A GPU with more than 20 GB of VRAM, for the conversion - The original google/gemma-4-31B-it weights, as the input to convert | Use case | Who | What they get back | | --- | --- | --- | | Custom bitrate quant | Quantizers | A quant made from the recipe at your target bitrate | | Quick test | Anyone curious | The ready made 3.00bpw quant on its branch | | Method comparison | Researchers | Numbers comparing uniform and optimized quants | ### How to set it up You need the exllamav3 scripts and the original Gemma weights, plus the files from this repository. - Python with pip - exllamav3 and its scripts - The original google/gemma-4-31B-it weights - A GPU with more than 20 GB of VRAM 1. Install the Hugging Face hub library. 2. Download the files from this repository. 3. Run the two steps below with your own paths and bitrate. ```sh pip install -U huggingface_hub hf download Dino0040/gemma-4-31b-it-exl3-optimized ``` ### How to use it First make a recipe for your target bitrate, which is nearly instant. Then convert the model with that recipe and the trace as calibration data, which takes an hour or two depending on the GPU. ```bash # 1. Recipe for your target bitrate (basically instant) python sc_optimize.py -m noise_attrib.json -rr rfn_2.00bpw.json -b -hb 6 -al 2.0 -mink 2 -ks 0.5 -maxk 4 -o recipe.yaml # 2. Convert with the recipe and the trace as calibration data (>20GB VRAM, An hour or two depending on GPU) python convert.py -i -o -w -rcp recipe.yaml -cd cal_trace.safetensors ``` | Tool | What it does | | --- | --- | | sc\_optimize.py | Builds a recipe for a target bitrate from the measurements | | convert.py | Converts the model using the recipe and the calibration trace | | -ks 0.5 | Allows half bit rates, and needs -maxk 4 with it | | -mink 2 | Keeps every tensor at 2 bits per weight or more | ### Example You pick a target of 3.5 bits per weight, replace the placeholder in the first command and run both steps. You end up with an EXL3 quant of Gemma 4 31B-it made from the optimized recipe. ### For newcomers and experts If you are new to this, a quant is a smaller copy of a model that trades some accuracy for less memory. If you only want to try one, use the 3.00bpw branch and skip the scripts. If you know the field, the files include a self sampled calibration trace, per tensor sensitivity measurements and error anchors from a 2 bpw probe. The recipe decides how many bits each tensor gets, and the trace is used as calibration data during conversion. Sensitivity measured on the full model → Recipe sets bits for each tensor → Convert uses recipe and trace → Optimized EXL3 quant results ### Good to know The author describes this as a one evening experiment out of curiosity, run on roughly five dollars of cloud time, and claims no expertise. The figures below are theirs, from a 3.00 bpw test against the bf16 model. Card test at 3.00 bpw, lower KL is better | Quant | Mean KL | Perplexity | | --- | --- | --- | | Uniform quant from July | 0.0561 | 1.2497 | | Optimized recipe | 0.0381 | 1.2276 | | Recipe plus trace calibration | 0.0301 | 1.2187 | The card puts the total drop in KL divergence at 46 percent at the same size as the old quant. - License: Apache 2.0 - Cost to run: none named for the files. Converting needs a GPU with more than 20 GB of VRAM - Data it reaches: the model weights and the calibration trace you point the scripts at - Not for: use as a ready to run model, since the repository holds recipe and measurement files ## README ````markdown # gemma-4-31b-it-exl3-optimized Holds the measurements and recipes for building an optimized EXL3 quant of Gemma 4 31B-it at any bitrate. For people who quantize models with exllamav3 and want to skip the slow measuring steps. Made by Dino0040 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository is not a ready model. It holds the pipeline outputs from exllamav3's optimized quantization recipes for google/gemma-4-31B-it. With these files you can build an optimized EXL3 quant at an arbitrary bitrate. The slow part, a sensitivity measurement of about an hour that needed roughly 120 GB of VRAM, is already done. A 3.00bpw quant is available on a separate branch for anyone who only wants to test it. Who it suits: People who quantize large models with exllamav3 and want better bit allocation without redoing the measurements. ```table Industry | Who uses it | How they use it | Example Model hosting | Engineers who serve large models | Build a quant at the bitrate that fits their GPU | Making a 3 bit quant that fits a single card Open model community | Quantization hobbyists | Reuse the measurements to try other bitrates | Generating a recipe for a different bits per weight target Research | People who study quantization error | Compare uniform and optimized bit allocation | Reproducing the card's KL divergence comparison ``` Skip it if you want a model to chat with right away, or if you have no GPU with more than 20 GB of VRAM for the conversion step. Where to use it: It works with the exllamav3 project and its scripts. The files download on Windows, macOS or Linux. - exllamav3: its optimize and convert scripts use these files - A GPU with more than 20 GB of VRAM, for the conversion - The original google/gemma-4-31B-it weights, as the input to convert ```table Use case | Who | What they get back Custom bitrate quant | Quantizers | A quant made from the recipe at your target bitrate Quick test | Anyone curious | The ready made 3.00bpw quant on its branch Method comparison | Researchers | Numbers comparing uniform and optimized quants ``` How to set it up: You need the exllamav3 scripts and the original Gemma weights, plus the files from this repository. - Python with pip - exllamav3 and its scripts - The original google/gemma-4-31B-it weights - A GPU with more than 20 GB of VRAM 1. Install the Hugging Face hub library. 2. Download the files from this repository. 3. Run the two steps below with your own paths and bitrate. ```sh pip install -U huggingface_hub hf download Dino0040/gemma-4-31b-it-exl3-optimized ``` How to use it: First make a recipe for your target bitrate, which is nearly instant. Then convert the model with that recipe and the trace as calibration data, which takes an hour or two depending on the GPU. ```bash # 1. Recipe for your target bitrate (basically instant) python sc_optimize.py -m noise_attrib.json -rr rfn_2.00bpw.json -b -hb 6 -al 2.0 -mink 2 -ks 0.5 -maxk 4 -o recipe.yaml # 2. Convert with the recipe and the trace as calibration data (>20GB VRAM, An hour or two depending on GPU) python convert.py -i -o -w -rcp recipe.yaml -cd cal_trace.safetensors ``` ```table Tool | What it does sc_optimize.py | Builds a recipe for a target bitrate from the measurements convert.py | Converts the model using the recipe and the calibration trace -ks 0.5 | Allows half bit rates, and needs -maxk 4 with it -mink 2 | Keeps every tensor at 2 bits per weight or more ``` Example: You pick a target of 3.5 bits per weight, replace the placeholder in the first command and run both steps. You end up with an EXL3 quant of Gemma 4 31B-it made from the optimized recipe. For newcomers and experts: If you are new to this, a quant is a smaller copy of a model that trades some accuracy for less memory. If you only want to try one, use the 3.00bpw branch and skip the scripts. If you know the field, the files include a self sampled calibration trace, per tensor sensitivity measurements and error anchors from a 2 bpw probe. The recipe decides how many bits each tensor gets, and the trace is used as calibration data during conversion. ```flow Sensitivity measured on the full model Recipe sets bits for each tensor Convert uses recipe and trace Optimized EXL3 quant results ``` Good to know: The author describes this as a one evening experiment out of curiosity, run on roughly five dollars of cloud time, and claims no expertise. The figures below are theirs, from a 3.00 bpw test against the bf16 model. ```table Title: Card test at 3.00 bpw, lower KL is better Quant | Mean KL | Perplexity Uniform quant from July | 0.0561 | 1.2497 Optimized recipe | 0.0381 | 1.2276 Recipe plus trace calibration | 0.0301 | 1.2187 ``` The card puts the total drop in KL divergence at 46 percent at the same size as the old quant. - License: Apache 2.0 - Cost to run: none named for the files. Converting needs a GPU with more than 20 GB of VRAM - Data it reaches: the model weights and the calibration trace you point the scripts at - Not for: use as a ready to run model, since the repository holds recipe and measurement files ## On Hugging Face, 13 November 2026 - Downloads in the last 30 days: 41 - Likes: 2 - License: apache-2.0 - Based on: google/gemma-4-31B-it - Tags: exl3 ## Links - Model: https://huggingface.co/Dino0040/gemma-4-31b-it-exl3-optimized - Files: https://huggingface.co/Dino0040/gemma-4-31b-it-exl3-optimized/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download Dino0040/gemma-4-31b-it-exl3-optimized ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [Dino0040/gemma-4-31b-it-exl3-optimized on Hugging Face](https://huggingface.co/Dino0040/gemma-4-31b-it-exl3-optimized) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | gemma-4-31b-it-exl3-optimized (this page) | 41 | 2 | — | apache-2.0 | 2026-10-12 | | [DeepSeek-V4.1-Flash-1017 EXL3 EAQ v1 2.9 bpw](https://mindket.com/p/e2aa9049-7d40-46f6-b187-57f9ffaaccf1) | 8,113 | 46 | 105B | mit | 2026-10-20 | | [Qwen3.8-27B-exl3](https://mindket.com/p/d2e0df1d-c113-4645-8870-2ce9b0728428) | 5,032 | 203 | — | apache-2.0 | 2026-08-27 | | [GLM-5.3-Flash EXL3 4bpw for TensorFold](https://mindket.com/p/dc92b67d-d498-4170-8472-86e468d8b950) | 4,746 | 73 | 87.8B | apache-2.0 | 2026-10-03 | | [DeepSeek-V4.1-Flash-EXL3-3bpw-2x-RTX-PRO-6000](https://mindket.com/p/eb19ba19-7dae-4d1c-874c-c0af3a0d8b4e) | 2,514 | 42 | 219B | mit | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017-exl3](https://mindket.com/p/93187f0c-ef38-4d6a-b054-9370ad7daaed) | 581 | 30 | — | mit | 2026-11-02 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-12 - First published: 2026-10-12 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open Dino0040/gemma-4-31b-it-exl3-optimized on Hugging Face](https://huggingface.co/Dino0040/gemma-4-31b-it-exl3-optimized) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e67fede3-4f93-4668-b2c9-e75092956dfe) - [Its page on Mindket](https://mindket.com/p/e67fede3-4f93-4668-b2c9-e75092956dfe)