Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF

Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF

Writes fiction with a tint of horror. GGUF builds of Gemma 3 12B instruct, quantized with an imatrix made from a horror model, for creative writers who run models locally.

Made by DavidAU on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a set of GGUF quantizations of google/gemma-3-12b-it by DavidAU. Two choices set it apart. The quantization uses an importance matrix built with the author's Grand Horror 16B model, which adds a tint of horror to the output. And the embed and output tensors stay at full BF16 precision in every quant, which the author calls maxed. The context length is 128k.

Who it suits: Writers and hobbyists who want a local model that leans toward dark, horror flavoured fiction, and who are comfortable picking a quant size.

Industry | Who uses it | How they use it | Example
Creative writing | Fiction writers | Draft scenes and story ideas with a horror tint | Asking for the opening of a ghost story
Game writing | Tabletop and game designers | Generate atmospheric descriptions and hooks | Drafting a haunted house room description
Hobby use | People who run local models | Try different quant sizes and compare the tint | Comparing an IQ4 file with a Q6 file on one prompt

Skip it if you want neutral general answers, since the tint is built into the quantization.

Where to use it: GGUF files load in local apps and runtimes that read that format. The files download on Windows, macOS or Linux.

• Local runtimes and apps that load GGUF files
• Windows, macOS or Linux: where you can fetch the files with Python
• Phones and other small devices: the card points to the Q4_0 and Q5_0 quants for these

Use case | Who | What they get back
Horror flavoured fiction | Writers | Story text with a dark tint
General chat | Everyday users | Normal replies, best with the larger quants
Portable use | Phone and device users | A smaller quant that fits limited hardware

How to set it up: You need Python to download, a GGUF capable app, and room for the quant you pick. The 4-bit size is about 5.9 GB, as an estimate.

• Python with pip
• A GGUF capable local app
• Disk space for one quant file

1. Install the Hugging Face hub library.
2. Download the files, or pick one quant file from the repository.
3. Load it in your app and set the context length.

pip install -U huggingface_hub
hf download DavidAU/Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF

How to use it: Pick a quant that matches your goal, then prompt as usual. The card also offers an optional system prompt that you paste exactly as shown, and a separate guide to parameters and samplers.

Quant types | What the card suggests
IQ3, IQ4XS, IQ4NL, Q4 | Best for creative work, with the horror tint strongest at the IQ4 types
Q5, Q6, Q8 | General use
Q4_0, Q5_0 | Portable devices and phones
IQ1, IQ2 | Smallest files, IQ1 is acceptable and IQ2 is usable

Example: You load an IQ4XS file and ask for the opening paragraph of a story set in an empty lighthouse. The reply is a short piece of fiction with a darker tone than the plain Gemma 3 model would likely give.

For newcomers and experts: If you are new to this, start with an IQ4XS or Q4 file, since the card says the horror tint is strongest there. Try one short story prompt, then compare it with a Q5 file.

If you know the field, the importance matrix was built with a horror model, and Q8 is a maxed quant only because an imatrix has no effect at that size. The author reports about 30 tokens per second at IQ4XS on a mid level graphics card.

Gemma 3 12B instruct base
Horror importance matrix applied
Quantized with maxed tensors
You pick a quant and write

Good to know: The card says the model follows Gemma 3 and points to a guide for the best parameter and sampler settings.

• License: Gemma
• Cost to run: none named, you supply the hardware
• Data it reaches: the card names none
• Not for: neutral general use, where the card suggests the Q5, Q6 and Q8 quants

On Hugging Face, 13 November 2026

• Task: text generation
• Parameters: 11.8B
• Weights: about 5.9 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 2,805
• Likes: 19
• License: gemma
• Based on: google/gemma-3-12b-it
• Tags: gguf, gemma3, instruct, horror, text-generation, imatrix, conversational

Links

• Model: https://huggingface.co/DavidAU/Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF
• Files: https://huggingface.co/DavidAU/Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download DavidAU/Gemma-3-12b-it-MAX-HORROR-Imatrix-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.