Llama-3.2-3B-Instruct-GGUF by unsloth

Llama-3.2-3B-Instruct-GGUF by unsloth

Chats, summarizes and answers questions with Meta's compact Llama 3.2 3B instruct model, in GGUF files by Unsloth from 2-bit up to 16-bit. Small enough for ordinary laptops.

Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF versions of Llama-3.2-3B-Instruct, a 3.2B parameter text model from Meta tuned to follow instructions. Unsloth made the files, and they come in sizes from 2-bit up to 16-bit. At 4-bit the weights are estimated at about 1.6 GB.

Who it suits: People who want a small local assistant for chat, summaries and retrieval help on a laptop or desktop. Developers who plan to fine-tune a model will also like the free notebooks the card points to.

Industry | Who uses it | How they use it | Example
Software | Developers | Add a small local chat model to an app | Draft short replies inside a tool
Education | Students | Chat with a model on a modest laptop | Ask for a plain explanation of a topic
Business | Small teams | Summarize notes and look things up in their own texts | Shorten a long meeting memo
Research | AI researchers | Fine-tune a compact model on their own data | Adapt it to a narrow task

People who need top answer quality on hard tasks can skip it, because this is a compact model.

Where to use it:

• Tools that load GGUF files, such as llama.cpp
• Unsloth's free notebooks, when you want to fine-tune the model
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Chat | Anyone | Written replies in a back and forth conversation
Summarizing | Students and teams | Shorter versions of longer texts
Agentic retrieval | Developers | Help finding and using information in tools
Fine-tuning | ML engineers | A model adapted to your own data

How to set it up:

• Python, for the Hugging Face hub tools
• A runner that loads GGUF files
• Disk space for the file you choose, with the 4-bit weights estimated at about 1.6 GB

1. Install the Hugging Face hub tools with pip.
2. Download the files with the hf command.
3. Load a file in your runner.

pip install -U huggingface_hub
hf download unsloth/Llama-3.2-3B-Instruct-GGUF

How to use it: The card shows no run command in the part given. Load the GGUF file in your runner and chat with it. The model was tuned for dialogue in several languages, so you can ask questions, request summaries or try retrieval style tasks.

Tool | What it does
hf download | Fetches the model files to your computer
GGUF runner | Loads a file and lets you chat with the model
Unsloth notebooks | Free notebooks for fine-tuning, with export to GGUF

Example: You download the 4-bit file, load it in your runner and paste in a long note. You ask for a short summary, and the model writes a few sentences back.

For newcomers and experts: Start with the 4-bit file, which is the smallest practical size for most laptops. If answers feel weak, try a larger file. If you want a model for your own data, use a notebook from Unsloth's collection.

The model is an auto-regressive language model with an optimized transformer design. Meta tuned it with supervised fine-tuning and reinforcement learning from human feedback, aiming for helpfulness and safety. The card says the instruction-tuned versions beat many open and closed chat models on common benchmarks.

Download a GGUF file
Load it in your runner
Send a prompt as a chat message
Read the reply

Good to know:

• Languages: Eight are officially supported, among them English, German, French, Spanish and Thai. More can be added by fine-tuning, within Meta's terms.
• Sizes: The files range from 2-bit to 16-bit, so you can trade quality for memory.
• Speed: The card says Unsloth notebooks fine-tune the Llama 3.2 3B model 2.4x faster with 58% less memory.
• License: The Llama 3.2 Community License, plus Meta's Acceptable Use Policy.
• Cost to run: none named. The notebooks run on free Google Colab, according to the card.
• Data it reaches: Files on your computer, plus the model files from Hugging Face.
• Not for: Uses that break Meta's Acceptable Use Policy.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 3.2B
• Weights: about 1.6 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 243,788
• Likes: 93
• License: llama3.2
• Languages: en
• Based on: meta-llama/Llama-3.2-3B-Instruct
• Quantized versions: 1
• Fine-tunes: 1
• Tags: transformers, gguf, llama, text-generation, llama-3, meta, facebook, unsloth, conversational

Links

• Model: https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-GGUF
• Files: https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download unsloth/Llama-3.2-3B-Instruct-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.