--- title: "Arete-27B-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/587591c7-2295-48cc-8bfc-5d3fd980cbfc" updated: "2026-11-12" --- # Arete-27B-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Nous Research's Arete 27B, a Qwen3.8-27B fine-tune that reads text and images, from GGUF files in six sizes with measured quality losses. For people running big models on their own GPU. Arete-27B-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by AtomicChat ## About ### What it is Arete-27B-GGUF holds quantized GGUF builds of Arete-27B, which is Nous Research's fine-tune of Qwen3.8-27B. AtomicChat made the builds from Nous's original weights with its own importance matrix. The card says Nous trained the model to change how it answers politically sensitive questions, and points to Nous's card for the details. ### Who it suits People who run large language models locally and want a GGUF file sized to their graphics card, with measured numbers for each size. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software development | Local-model users | Run a 27B model offline on their own GPU | A local chat model on a 24 GB card | | Research | Model researchers | Compare quantization levels on one model | Reading the KL divergence of each file | | AI tooling | App builders | Serve the model through llama.cpp | A local server answering chat requests | | Education | Students | Study how quantization trades size for quality | Comparing a 16.2 GB file with a 28.6 GB one | If your graphics card has under about 16 GB of memory, you can skip it. ### Where to use it - llama.cpp, through its llama-server program - Atomic Chat, the maker's app for running open-weight models locally and offline - Windows, macOS or Linux, after you fetch the files with Python | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Individuals | Answers from a 27B model without sending data away | | Image reading | Individuals | Descriptions of images, with one extra vision file | | Quantization study | Researchers | Size, KL divergence and top-1 agreement per file | ### How to set it up - Python and pip - Enough disk space and graphics memory for the file you pick, from 13.6 GB to 28.6 GB - llama.cpp, or another app that loads GGUF files 1. Install the Hugging Face hub tools. 2. Download the model files. 3. Pick a file that fits your card, as the card's table advises, and start a server with it. ```sh pip install -U huggingface_hub hf download AtomicChat/Arete-27B-GGUF ``` ### How to use it Start llama-server with the file you chose. The chat template is stored inside every file, and llama.cpp applies it by itself. Thinking is on by default, and you can turn it off or shorten it with a template option. ```bash llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 ``` ```bash # no thinking llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 --chat-template-kwargs '{"enable_thinking": false}' # shorter thinking llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 --chat-template-kwargs '{"reasoning_effort": "low"}' ``` The reasoning effort can be xhigh, which is the default, medium or low. The value high makes the template stop with an error. | Tool | What it does | | --- | --- | | llama-server | Serves the chosen GGUF file for chat | | enable\_thinking | Template option that turns thinking off when set to false | | reasoning\_effort | Template option that sets how long the model thinks | ### Example You start llama-server with the Q4\_K\_M file on a 24 GB card and send it a chat question. With thinking on, the model reasons first and then answers. Switching thinking off gives a direct answer. ### For newcomers and experts If you are new to GGUF, match the file to your graphics memory using the card's table, then start llama-server and chat. If two neighboring files both fit, the card suggests taking the larger one. The card's numbers are measured, not estimated. KL divergence measures how much a file's output distribution shifts from the full model. Top-1 is the share of times the file and the full model choose the same next word. The AD files use an Atomic Dynamic layout, named after the types of the two largest tensor groups. Each token of context costs the model 64 KB of attention cache, so plan memory for the context as well as the file. It has no multi-token prediction head. Pick a file that fits your card → Start llama-server with it → The chat template applies itself → Chat, with thinking on or off ### Good to know - Sizes: the files run from 13.6 GB to 28.6 GB, and a larger context needs more memory - Quality: smaller files drift further from the original weights Top-1 agreement with the original weights, in percent | Label | Value | | --- | --- | | Q8\_0 | 98.87% | | AD-Q6\_K | 98.58% | | AD-Q5\_K\_M | 97.48% | | AD-Q4\_K\_M | 95.89% | | AD-IQ4\_XS | 95.60% | | AD-IQ3\_S | 92.31% | - License: Apache-2.0 - Cost to run: no API key or paid service is named, only your own GPU - Data it reaches: downloading fetches model files, and chats run locally on your machine - Not for: cards under about 16 GB, and clients that send a reasoning effort of high, which gets no reply ## README ````markdown # Arete-27B-GGUF Runs Nous Research's Arete 27B, a Qwen3.8-27B fine-tune that reads text and images, from GGUF files in six sizes with measured quality losses. For people running big models on their own GPU. Made by AtomicChat on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Arete-27B-GGUF holds quantized GGUF builds of Arete-27B, which is Nous Research's fine-tune of Qwen3.8-27B. AtomicChat made the builds from Nous's original weights with its own importance matrix. The card says Nous trained the model to change how it answers politically sensitive questions, and points to Nous's card for the details. Who it suits: People who run large language models locally and want a GGUF file sized to their graphics card, with measured numbers for each size. ```table Industry | Who uses it | How they use it | Example Software development | Local-model users | Run a 27B model offline on their own GPU | A local chat model on a 24 GB card Research | Model researchers | Compare quantization levels on one model | Reading the KL divergence of each file AI tooling | App builders | Serve the model through llama.cpp | A local server answering chat requests Education | Students | Study how quantization trades size for quality | Comparing a 16.2 GB file with a 28.6 GB one ``` If your graphics card has under about 16 GB of memory, you can skip it. Where to use it: - llama.cpp, through its llama-server program - Atomic Chat, the maker's app for running open-weight models locally and offline - Windows, macOS or Linux, after you fetch the files with Python ```table Use case | Who | What they get back Local chat | Individuals | Answers from a 27B model without sending data away Image reading | Individuals | Descriptions of images, with one extra vision file Quantization study | Researchers | Size, KL divergence and top-1 agreement per file ``` How to set it up: - Python and pip - Enough disk space and graphics memory for the file you pick, from 13.6 GB to 28.6 GB - llama.cpp, or another app that loads GGUF files 1. Install the Hugging Face hub tools. 2. Download the model files. 3. Pick a file that fits your card, as the card's table advises, and start a server with it. ```sh pip install -U huggingface_hub hf download AtomicChat/Arete-27B-GGUF ``` How to use it: Start llama-server with the file you chose. The chat template is stored inside every file, and llama.cpp applies it by itself. Thinking is on by default, and you can turn it off or shorten it with a template option. ```bash llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 ``` ```bash # no thinking llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 --chat-template-kwargs '{"enable_thinking": false}' # shorter thinking llama-server -m Arete-27B-AD-Q4_K_M.gguf -ngl 99 -c 8192 --chat-template-kwargs '{"reasoning_effort": "low"}' ``` The reasoning effort can be xhigh, which is the default, medium or low. The value high makes the template stop with an error. ```table Tool | What it does llama-server | Serves the chosen GGUF file for chat enable_thinking | Template option that turns thinking off when set to false reasoning_effort | Template option that sets how long the model thinks ``` Example: You start llama-server with the Q4_K_M file on a 24 GB card and send it a chat question. With thinking on, the model reasons first and then answers. Switching thinking off gives a direct answer. For newcomers and experts: If you are new to GGUF, match the file to your graphics memory using the card's table, then start llama-server and chat. If two neighboring files both fit, the card suggests taking the larger one. The card's numbers are measured, not estimated. KL divergence measures how much a file's output distribution shifts from the full model. Top-1 is the share of times the file and the full model choose the same next word. The AD files use an Atomic Dynamic layout, named after the types of the two largest tensor groups. Each token of context costs the model 64 KB of attention cache, so plan memory for the context as well as the file. It has no multi-token prediction head. ```flow Pick a file that fits your card Start llama-server with it The chat template applies itself Chat, with thinking on or off ``` Good to know: - Sizes: the files run from 13.6 GB to 28.6 GB, and a larger context needs more memory - Quality: smaller files drift further from the original weights ```chart Title: Top-1 agreement with the original weights, in percent Q8_0: 98.87% AD-Q6_K: 98.58% AD-Q5_K_M: 97.48% AD-Q4_K_M: 95.89% AD-IQ4_XS: 95.60% AD-IQ3_S: 92.31% ``` - License: Apache-2.0 - Cost to run: no API key or paid service is named, only your own GPU - Data it reaches: downloading fetches model files, and chats run locally on your machine - Not for: cards under about 16 GB, and clients that send a reasoning effort of high, which gets no reply ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: gguf - Parameters: 26.9B - Weights: about 13.4 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 1,933 - Likes: 15 - License: apache-2.0 - Based on: NousResearch/Arete-27B - Tags: gguf, llama.cpp, arete, nousresearch, imatrix, quantized, conversational, atomic-chat, image-text-to-text ## Links - Model: https://huggingface.co/AtomicChat/Arete-27B-GGUF - Files: https://huggingface.co/AtomicChat/Arete-27B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download AtomicChat/Arete-27B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [AtomicChat/Arete-27B-GGUF on Hugging Face](https://huggingface.co/AtomicChat/Arete-27B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Arete-27B-GGUF (this page) | 1,933 | 15 | 26.9B | apache-2.0 | 2026-11-08 | | [Ripple-65B-Preview-GGUF](https://mindket.com/p/00143f02-5553-4372-8620-836dbb7dfec1) | 100,818 | 53 | 64.8B | apache-2.0 | 2026-11-06 | | [MiniCPM-V-4.7-1B-gguf](https://mindket.com/p/f793ef5c-72e1-400b-91fa-5b2c31b01f3f) | 168,636 | 114 | 1B | apache-2.0 | 2026-11-11 | | [Unlimited-OCR-2-GGUF](https://mindket.com/p/0d8a371b-6f90-4b59-8dd6-b520e35b577b) | 8,452 | 8 | 2.9B | mit | 2026-11-01 | | [Axion-1.0-35B-APEX-I-MiniPlus-V2.1-GGUF](https://mindket.com/p/c72e0389-3329-43ad-bfb2-f48f037bce1f) | 686 | 4 | 34.7B | apache-2.0 | 2026-11-05 | | [Groot-19B-A3B-TTQ-GGUF](https://mindket.com/p/6a828233-6f05-4485-bd2e-8181cc50d297) | 6,541 | 4 | 19B | apache-2.0 | 2026-11-06 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-08 - First published: 2026-11-08 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open AtomicChat/Arete-27B-GGUF on Hugging Face](https://huggingface.co/AtomicChat/Arete-27B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/587591c7-2295-48cc-8bfc-5d3fd980cbfc) - [Its page on Mindket](https://mindket.com/p/587591c7-2295-48cc-8bfc-5d3fd980cbfc)