granite-4.2-8b GGUF

granite-4.2-8b GGUF

Holds GGUF copies of IBM's Granite 4.2 8B model in several quantization levels, ready for llama.cpp. For people who want to run the model locally and read its details from the base model card.

Made by ibm-granite on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF conversions of IBM's granite-4.2-8b, made in several quantization levels. The model has about 8.8 billion parameters. The files are meant for llama.cpp, which loads GGUF models. The card is short and sends you to the base model's card for the full details.

Who it suits: It suits people who want to run a Granite model on their own computer through llama.cpp, and who are happy to read the base model card for what it can do.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists | Run an 8.8B model offline with llama.cpp | Chatting with a local model on a desktop
Software | App developers | Embed a local Granite model in a tool | A private text assistant inside an app
Research | ML researchers | Compare quantization levels of one base model | Checking speed against memory use

Skip it if you want details on training, languages or benchmarks, since this card leaves those to the base model.

Where to use it:

• llama.cpp, the library this repository is listed for
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Local text generation | Hobbyists | Replies from an 8.8B model on your own machine
Quant selection | Developers | A choice of file sizes to fit your memory

How to set it up: You need these first.

• Python with pip, to fetch the files
• A build of llama.cpp that can load the model
• Memory for the weights, which the record puts at roughly 4.4 GB once quantized to 4 bits

1. Install the Hugging Face hub library with pip.
2. Download the repository files with the hf command.
3. Choose one quantization file for your memory.
4. Load it in llama.cpp, following the base model's card for details.

pip install -U huggingface_hub
hf download ibm-granite/granite-4.2-8b-GGUF

How to use it: Load one GGUF file in llama.cpp and send it prompts. This card shows no run command of its own and points to the base model card for its full description.

hf download ibm-granite/granite-4.2-8b-GGUF

Example: Download the files, load one in llama.cpp and send a short prompt. You get generated text back. The card gives no sample prompt or output, so check the base model card for what the model is meant to do.

For newcomers and experts: If you are new, GGUF is a file format that lets llama.cpp run a model on your own computer. Start by downloading the files and reading the base model's card before you pick a size.

If you know the field, these are conversions of an IBM Granite base model into GGUF with several quantization levels. The card gives no benchmarks or per-quant measurements, so test the quant you choose on your own tasks.

IBM granite-4.2-8b base model
Converted to GGUF
Quantized at several levels
You pick one file
llama.cpp loads it

Good to know:

• Size: about 8.8B parameters, with about 4.4 GB at 4-bit by estimate
• Format: GGUF only
• Details: the base model card holds the full description

• License: Apache 2.0
• Cost to run: none named, the source lists no API key, account or paid service
• Data it reaches: the files you download and the prompts you give your own llama.cpp setup, and the source names no web or account access
• Not for: judging quality from this card, since it holds no benchmarks or usage notes

On Hugging Face, 13 November 2026

• Library: llama.cpp
• Parameters: 8.8B
• Weights: about 4.4 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 282,269
• Likes: 49
• License: apache-2.0
• Based on: ibm-granite/granite-4.2-8b
• Tags: llama.cpp, gguf, language, granite-4.2, conversational

Links

• Model: https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF
• Files: https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ibm-granite/granite-4.2-8b-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.