--- title: "Hermes-3-Llama-3.1-8B-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/8d818a45-a1d6-40c1-b241-b3edc81f29ed" updated: "2026-11-12" --- # Hermes-3-Llama-3.1-8B-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Hermes 3, an 8B general-purpose chat model from NousResearch, as a quantized GGUF file for llama.cpp. Suits people who want a local assistant with function calling, structured output and code help. Hermes-3-Llama-3.1-8B-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by NousResearch ## About ### What it is This is a GGUF copy of Hermes 3 8B, a general-purpose language model from Nous Research built on Meta-Llama-3.1-8B. The quantized file is meant for llama.cpp, so you can run the model on your own computer instead of calling a hosted service. The card lists better role play, reasoning, multi-turn chat, long context and code writing than Hermes 2, plus function calling and structured output. ### Who it suits It suits people who want a local chat model they can steer with a system prompt, and developers who need tool calling or structured replies from a small model. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a local assistant that writes and explains code | Draft a function offline | | AI tooling | Agent builders | Use function calling and structured output in an agent loop | Return a tool call as JSON | | Hobby projects | Tinkerers | Chat and role play on a home machine | Talk to a custom persona | | Research | Students | Try a Llama 3.1 based model without a paid API | Compare replies on a test prompt | People who only need a hosted chat app, or who cannot spare about 4 GB of memory for the weights, can skip it. ### Where to use it The file runs wherever llama.cpp runs, and you fetch it from a computer with Python. - Windows, macOS and Linux desktops - llama.cpp on the command line or in a local server - Apps that load GGUF files and speak the ChatML format - Agent code that talks to an OpenAI style endpoint | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Anyone with a spare computer | Private replies with no hosted service | | Function calling | Agent developers | A structured call that code can read | | Code help | Programmers | Draft code and explanations | ### How to set it up You need a computer with Python, enough free disk space and memory for a file of about 4 GB, and llama.cpp to run the model. - Python with pip - llama.cpp installed - Room for the 4-bit weights, estimated at about 4 GB 1. Install the Hugging Face hub package. 2. Download the repository files to your computer. 3. Point llama.cpp at the GGUF file you want. ```sh pip install -U huggingface_hub hf download NousResearch/Hermes-3-Llama-3.1-8B-GGUF ``` ### How to use it Hermes 3 expects the ChatML prompt format, where special tokens mark the start and end of each turn and each turn has a role. A system message steers the rules, role and style. Without one, you leave that turn out. The card shows this Python snippet for building a chat prompt from messages. ```python messages = [ {"role": "system", "content": "You are Hermes 3."}, {"role": "user", "content": "Hello, who are you?"} ] gen_input = tokenizer.apply_chat_template(messages, return_tensors="pt") model.generate(**gen_input) ``` ### Example You set a system message that gives the model a role, then ask "Hello, who are you?". The model answers in character, as one assistant turn, and you can keep the chat going turn by turn. ### For newcomers and experts If you are new, start by downloading the files with the two commands above, then load the GGUF file in a program that supports it and send one short hello. A quantized file trades some accuracy for a much smaller size. If you know the field, this is the Hermes 3 fine-tune of Llama 3.1 8B in GGUF form. It uses ChatML, which maps cleanly to OpenAI style chat endpoints, and it was trained on specific system prompts for function calling. Download the GGUF repository → Load the file in llama.cpp → Send a ChatML prompt → Read the assistant reply ### Good to know The card says Hermes 3 is competitive with Llama 3.1 Instruct models, with varying strengths and weaknesses on each side. Treat that as a general guide and test it on your own task. - License: llama3 - Cost to run: none named, the weights are free to download and run on your own computer - Data it reaches: files you download to your computer, and nothing else is named - Not for: users who need the full BF16 weights, which are in a separate repository ## README ````markdown # Hermes-3-Llama-3.1-8B-GGUF Runs Hermes 3, an 8B general-purpose chat model from NousResearch, as a quantized GGUF file for llama.cpp. Suits people who want a local assistant with function calling, structured output and code help. Made by NousResearch on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a GGUF copy of Hermes 3 8B, a general-purpose language model from Nous Research built on Meta-Llama-3.1-8B. The quantized file is meant for llama.cpp, so you can run the model on your own computer instead of calling a hosted service. The card lists better role play, reasoning, multi-turn chat, long context and code writing than Hermes 2, plus function calling and structured output. Who it suits: It suits people who want a local chat model they can steer with a system prompt, and developers who need tool calling or structured replies from a small model. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a local assistant that writes and explains code | Draft a function offline AI tooling | Agent builders | Use function calling and structured output in an agent loop | Return a tool call as JSON Hobby projects | Tinkerers | Chat and role play on a home machine | Talk to a custom persona Research | Students | Try a Llama 3.1 based model without a paid API | Compare replies on a test prompt ``` People who only need a hosted chat app, or who cannot spare about 4 GB of memory for the weights, can skip it. Where to use it: The file runs wherever llama.cpp runs, and you fetch it from a computer with Python. - Windows, macOS and Linux desktops - llama.cpp on the command line or in a local server - Apps that load GGUF files and speak the ChatML format - Agent code that talks to an OpenAI style endpoint ```table Use case | Who | What they get back Local chat | Anyone with a spare computer | Private replies with no hosted service Function calling | Agent developers | A structured call that code can read Code help | Programmers | Draft code and explanations ``` How to set it up: You need a computer with Python, enough free disk space and memory for a file of about 4 GB, and llama.cpp to run the model. - Python with pip - llama.cpp installed - Room for the 4-bit weights, estimated at about 4 GB 1. Install the Hugging Face hub package. 2. Download the repository files to your computer. 3. Point llama.cpp at the GGUF file you want. ```sh pip install -U huggingface_hub hf download NousResearch/Hermes-3-Llama-3.1-8B-GGUF ``` How to use it: Hermes 3 expects the ChatML prompt format, where special tokens mark the start and end of each turn and each turn has a role. A system message steers the rules, role and style. Without one, you leave that turn out. The card shows this Python snippet for building a chat prompt from messages. ```python messages = [ {"role": "system", "content": "You are Hermes 3."}, {"role": "user", "content": "Hello, who are you?"} ] gen_input = tokenizer.apply_chat_template(messages, return_tensors="pt") model.generate(**gen_input) ``` Example: You set a system message that gives the model a role, then ask "Hello, who are you?". The model answers in character, as one assistant turn, and you can keep the chat going turn by turn. For newcomers and experts: If you are new, start by downloading the files with the two commands above, then load the GGUF file in a program that supports it and send one short hello. A quantized file trades some accuracy for a much smaller size. If you know the field, this is the Hermes 3 fine-tune of Llama 3.1 8B in GGUF form. It uses ChatML, which maps cleanly to OpenAI style chat endpoints, and it was trained on specific system prompts for function calling. ```flow Download the GGUF repository Load the file in llama.cpp Send a ChatML prompt Read the assistant reply ``` Good to know: The card says Hermes 3 is competitive with Llama 3.1 Instruct models, with varying strengths and weaknesses on each side. Treat that as a general guide and test it on your own task. - License: llama3 - Cost to run: none named, the weights are free to download and run on your own computer - Data it reaches: files you download to your computer, and nothing else is named - Not for: users who need the full BF16 weights, which are in a separate repository ## On Hugging Face, 13 November 2026 - Parameters: 8B - Weights: about 4 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 69,009 - Likes: 229 - License: llama3 - Languages: en - Based on: meta-llama/Meta-Llama-3.1-8B - Tags: gguf, llama-3, instruct, finetune, chatml, gpt4, distillation, axolotl, roleplaying, chat, conversational ## Links - Model: https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF - Files: https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download NousResearch/Hermes-3-Llama-3.1-8B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [NousResearch/Hermes-3-Llama-3.1-8B-GGUF on Hugging Face](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Hermes-3-Llama-3.1-8B-GGUF (this page) | 69,009 | 229 | 8B | llama3 | 2024-08-23 | | [Hermes 3 Llama-3.1 8B](https://mindket.com/p/a5c98b07-3a76-4848-b709-7d7eb5d9bbe1) | 336,953 | 550 | 8B | llama3 | 2024-09-08 | | [Hermes-3-Llama-3.2-3B](https://mindket.com/p/cf2abfc3-ba62-4495-b29c-7142ffe7f877) | 4,602 | 187 | 3.2B | llama3 | 2024-12-18 | | [Hermes-4.3-36B-GGUF](https://mindket.com/p/3bbe6dee-1cc7-4847-8e5f-0007d484fb94) | 28,565 | 99 | 36.2B | apache-2.0 | 2025-12-03 | | [Hermes-4.3-36B](https://mindket.com/p/93046541-b0ed-48e3-bed3-484d63046d91) | 2,559 | 321 | 36.2B | apache-2.0 | 2025-12-06 | | [Hermes 4 14B](https://mindket.com/p/22421189-6edb-42cc-b1d5-ac58c4c31726) | 6,062 | 205 | 425K | apache-2.0 | 2026-01-09 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Bluesky · @aichina.news · 2026-04-17](https://bsky.app/profile/aichina.news/post/3mjpd6f64z52t): Hermes 3 achieves 70B-class reasoning in an 8B package for Ascend developers. This GGUF release brings top-tier steerability and 128k context to Huawei's ecosystem, marking a major leap beyond standard quantisation. https://aichina.news/models/NousResearch/Hermes-3-Llama-3.1-8B-GGUF/ (248... ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2024-08-23 - First published: 2024-08-05 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open NousResearch/Hermes-3-Llama-3.1-8B-GGUF on Hugging Face](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/8d818a45-a1d6-40c1-b241-b3edc81f29ed) - [Its page on Mindket](https://mindket.com/p/8d818a45-a1d6-40c1-b241-b3edc81f29ed)