Mellum2.1-12B-A2.5B-Thinking-GGUF

Mellum2.1-12B-A2.5B-Thinking-GGUF

Runs JetBrains' 12B reasoning model locally in llama.cpp, Ollama or LM Studio, with five GGUF sizes to pick from. For developers who want a thinking model with a 131,072-token context on their own machine.

Made by JetBrains on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Mellum2.1-12B-A2.5B-Thinking-GGUF holds JetBrains' reasoning model in the GGUF format, in five sizes. The model writes out its chain of thought between think tags before it gives the final answer. The repository also carries an MTP head that speeds up generation in llama.cpp.

Who it suits: Developers and tinkerers who run models locally with llama.cpp, Ollama or LM Studio and want a reasoning model with a 131,072-token context. The files carry the Apache 2.0 license.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a reasoning model offline behind a local server | Pointing a chat client at a local endpoint
Education | Students and teachers | Ask for worked reasoning on logic or math questions | Asking whether a number is a power of 2
Research | Quantization researchers | Compare file size against closeness to the 16-bit original | Reading the size and agreement figures per quantization

Anyone who wants instant short answers can skip it, because the model spends tokens on reasoning before it replies.

Where to use it:

• llama.cpp, through its server and command-line tool
• Ollama and LM Studio, which the card names as runtimes
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Local chat | Developers | A reasoning trace followed by a final answer
Local API | App builders | A server that other programs can call like a hosted model
Faster generation | Tinkerers | Quicker output when the MTP draft file is accepted

How to set it up:

• Python and the huggingface_hub package, to download the files
• A GGUF runtime such as llama.cpp, Ollama or LM Studio
• Room for one file, from 7.0 GB to 24.3 GB depending on the quantization you pick

1. Install Python and a GGUF runtime if you do not have them yet.
2. Install the huggingface_hub package with the first command below.
3. Download the one quantization you want with the second command. The card recommends Q4_K_M.
4. Run it with llama.cpp, using the commands in the next section.

pip install -U huggingface_hub
hf download JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf --local-dir .

Quantization | Size | Top-token match with BF16
Q8_0 | 12.9 GB | 96.1 percent
Q6_K | 10.9 GB | 93.9 percent
Q4_K_M | 8.1 GB | 88.0 percent
MXFP4_MOE | 7.0 GB | 85.6 percent

How to use it: The first command below fetches the chosen quantization and serves it. The second runs a single prompt from a local file. Both use the full 131,072-token context and the sampling settings the card gives.

# Pull and serve in one step (downloads the GGUF automatically; the suffix picks the quantization)
llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M \
  --ctx-size 131072 \
  --temp 0.6 --top-p 0.95 --top-k 20

# Or run a one-off prompt with a local file
llama-cli -m Mellum2.1-12B-A2.5B-Thinking-Q4_K_M.gguf \
  --ctx-size 131072 \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -p "Is 1024 a power of 2? Explain your reasoning."

Tool | What it does
llama-server | Starts a local server and fetches the file you name by its quantization suffix
llama-cli | Runs one prompt against a local file and prints the reply
--ctx-size 131072 | Sets the context to the model's full length
--spec-type draft-mtp | Uses the MTP head as a draft, so the full model only has to verify tokens

Example: You run the llama-cli command and ask whether 1000 is a power of 2, with a request to explain why. The model prints its reasoning first, then a short answer.

For newcomers and experts: A reasoning model thinks out loud before it answers, so replies take longer but show their steps. A good first step is the Q4_K_M download and the llama-cli command.

It is a Mixture-of-Experts model with 64 experts and 8 active per token. The MTP head is one extra block that guesses the next token after the one just sampled. The full model checks each guess, so output does not change, and speed rises when guesses are accepted.

Pick a quantization
Download the one file
Run it in llama.cpp, Ollama or LM Studio
Model thinks inside think tags
Final answer follows

Good to know:

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the download fetches files from Hugging Face, and the server listens on your own computer
• The closeness figures were measured against the BF16 file on Wikitext-2 at a 512-token context
• The MTP head comes as a 1.8 GB draft file for every quantization, or as a 25.2 GB single file with the model

Title: File size by quantization in GB
BF16: 24.3 GB
Q8_0: 12.9 GB
Q6_K: 10.9 GB
Q4_K_M: 8.1 GB
MXFP4_MOE: 7.0 GB

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 12.1B
• Weights: about 6.1 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 422,635
• Likes: 172
• License: apache-2.0
• Languages: en
• Based on: JetBrains/Mellum2.1-12B-A2.5B-Thinking
• Tags: gguf, mellum, llama.cpp, quantized, thinking, speculative-decoding, text-generation, conversational

Links

• Model: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF
• Files: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.