Hermes-3-Llama-3.1-8B-GGUF

Hermes-3-Llama-3.1-8B-GGUF

Runs Hermes 3, an 8B general-purpose chat model from NousResearch, as a quantized GGUF file for llama.cpp. Suits people who want a local assistant with function calling, structured output and code help.

Made by NousResearch on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a GGUF copy of Hermes 3 8B, a general-purpose language model from Nous Research built on Meta-Llama-3.1-8B. The quantized file is meant for llama.cpp, so you can run the model on your own computer instead of calling a hosted service.

The card lists better role play, reasoning, multi-turn chat, long context and code writing than Hermes 2, plus function calling and structured output.

Who it suits: It suits people who want a local chat model they can steer with a system prompt, and developers who need tool calling or structured replies from a small model.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local assistant that writes and explains code | Draft a function offline
AI tooling | Agent builders | Use function calling and structured output in an agent loop | Return a tool call as JSON
Hobby projects | Tinkerers | Chat and role play on a home machine | Talk to a custom persona
Research | Students | Try a Llama 3.1 based model without a paid API | Compare replies on a test prompt

People who only need a hosted chat app, or who cannot spare about 4 GB of memory for the weights, can skip it.

Where to use it: The file runs wherever llama.cpp runs, and you fetch it from a computer with Python.

• Windows, macOS and Linux desktops
• llama.cpp on the command line or in a local server
• Apps that load GGUF files and speak the ChatML format
• Agent code that talks to an OpenAI style endpoint

Use case | Who | What they get back
Local chat | Anyone with a spare computer | Private replies with no hosted service
Function calling | Agent developers | A structured call that code can read
Code help | Programmers | Draft code and explanations

How to set it up: You need a computer with Python, enough free disk space and memory for a file of about 4 GB, and llama.cpp to run the model.

• Python with pip
• llama.cpp installed
• Room for the 4-bit weights, estimated at about 4 GB

1. Install the Hugging Face hub package.
2. Download the repository files to your computer.
3. Point llama.cpp at the GGUF file you want.

pip install -U huggingface_hub
hf download NousResearch/Hermes-3-Llama-3.1-8B-GGUF

How to use it: Hermes 3 expects the ChatML prompt format, where special tokens mark the start and end of each turn and each turn has a role. A system message steers the rules, role and style. Without one, you leave that turn out.

The card shows this Python snippet for building a chat prompt from messages.

messages = [
    {"role": "system", "content": "You are Hermes 3."},
    {"role": "user", "content": "Hello, who are you?"}
]
gen_input = tokenizer.apply_chat_template(messages, return_tensors="pt")
model.generate(**gen_input)

Example: You set a system message that gives the model a role, then ask "Hello, who are you?". The model answers in character, as one assistant turn, and you can keep the chat going turn by turn.

For newcomers and experts: If you are new, start by downloading the files with the two commands above, then load the GGUF file in a program that supports it and send one short hello. A quantized file trades some accuracy for a much smaller size.

If you know the field, this is the Hermes 3 fine-tune of Llama 3.1 8B in GGUF form. It uses ChatML, which maps cleanly to OpenAI style chat endpoints, and it was trained on specific system prompts for function calling.

Download the GGUF repository
Load the file in llama.cpp
Send a ChatML prompt
Read the assistant reply

Good to know: The card says Hermes 3 is competitive with Llama 3.1 Instruct models, with varying strengths and weaknesses on each side. Treat that as a general guide and test it on your own task.

• License: llama3
• Cost to run: none named, the weights are free to download and run on your own computer
• Data it reaches: files you download to your computer, and nothing else is named
• Not for: users who need the full BF16 weights, which are in a separate repository

On Hugging Face, 13 November 2026

• Parameters: 8B
• Weights: about 4 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 69,009
• Likes: 229
• License: llama3
• Languages: en
• Based on: meta-llama/Meta-Llama-3.1-8B
• Tags: gguf, llama-3, instruct, finetune, chatml, gpt4, distillation, axolotl, roleplaying, chat, conversational

Links

• Model: https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF
• Files: https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download NousResearch/Hermes-3-Llama-3.1-8B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.