Qwen3.8-27B Agention Precision GGUF

Qwen3.8-27B Agention Precision GGUF

Packs Qwen3.8-27B as llama.cpp GGUF files in eight sizes from 8.95 to 16.35 GiB, plus a vision projector. The author reports less drift from the BF16 model than other quants at each size.

Made by agentionai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds Agention Precision quants of Qwen3.8-27B in GGUF form for llama.cpp. The author says each file uses a non-uniform assignment with error correction from a custom encoder, to stay close to the original BF16 model. A vision projector and an MTP draft head come in the pack.

Who it suits: It suits people who run a 27B model on a single consumer GPU and want to choose a file by their video memory.

Title: Who might use it
Industry | Who uses it | How they use it | Example
Local AI | Enthusiasts | Run a 27B model on a 12, 16 or 24 GB graphics card | Picking the file that fits their card
Software development | Developers | Serve a local model through llama.cpp | A coding assistant on their own machine
Model research | Researchers | Compare quants by how closely they track BF16 | Reading the KL divergence results on three text sets

People who want the original BF16 weights can skip it.

Where to use it:

• llama.cpp: the card says the files use standard types, with no fork and no flags
• GPUs with 12, 16 or 24 GB of memory, depending on the file
• Windows, macOS or Linux: download the files with Python

Title: Jobs it does
Use case | Who | What they get back
Text chat and coding | Developers | Replies from a quantized Qwen3.8-27B
Image questions | Local AI users | Image input through the vision projector file
Quant comparison | Researchers | Files to measure against the BF16 model

How to set it up: You need these first:

• Python on Windows, macOS or Linux, to download the files
• A llama.cpp build that can load GGUF files
• A GPU with room for the file you pick plus its context

1. Install the Hugging Face hub package.
2. Download the repository files.
3. Pick one GGUF file by your video memory, and add the vision projector if you want images.

pip install -U huggingface_hub
hf download agentionai/Qwen3.8-27B-AP-GGUF

Title: Which file for which card
File | Tier the card lists | Fits
AP-Q4_K_XL | Most headroom | 24 GB
AP-Q4_K_M | 24 GB, smaller | 24 GB
AP-IQ4_XS | Default | 16 GB
AP-Q3_K_XL | Need headroom | 16 GB with longer context
AP-IQ3_S | 12 GB, more quality | 16 GB at 8 to 16k context
AP-IQ3_XS | 12 GB, balanced | 12 GB at 8 to 16k context
AP-IQ3_XXS | 12 GB, multi-model | 12 GB at 8 to 16k context
AP-IQ2_S | 12 GB, smallest | 12 GB at 8 to 16k context
mmproj-BF16.gguf | Vision | Any tier

How to use it: Load your chosen file in llama.cpp the way you load any GGUF. The card advises AP-IQ4_XS if it fits and a step down only when memory is short. Only a quarter of the layers use full attention, so a 32k context costs about 1 GiB at q8_0.

Example: You have a 16 GB graphics card, so you load AP-IQ4_XS and ask it a question. For a photo question, you add the vision projector file. The card reports AP-IQ4_XS as 8% closer to BF16 than the leading quant of its size.

For newcomers and experts: If you are new to this, a quant is a smaller copy of a model, and this pack offers several sizes. Start with AP-IQ4_XS and move to a smaller file only if it does not fit.

If you work in the field, fidelity is the KL divergence of the next-token distribution against Qwen3.8-27B in BF16, measured on 60 chunks of 2,048 tokens over three corpora. They are a held-out technical set, a web slice and wikitext-2. The held-out set is internal and unpublished, so its column cannot be re-run. The card gives the llama-perplexity commands and says every compared file was measured the same way.

Convert Qwen3.8-27B to a BF16 GGUF
Record BF16 logits for each corpus
Score every quant against those logits
Compare mean KLD and worst-1% KLD

Good to know: The card lists eight file sizes.

Title: File size in GiB
AP-Q4_K_XL: 16.35 GiB
AP-Q4_K_M: 15.33 GiB
AP-IQ4_XS: 13.27 GiB
AP-Q3_K_XL: 12.24 GiB
AP-IQ3_S: 11.21 GiB
AP-IQ3_XS: 10.70 GiB
AP-IQ3_XXS: 10.00 GiB
AP-IQ2_S: 8.95 GiB

• License: apache-2.0
• Cost to run: none named
• Data it reaches: only the prompts and images you give it, run locally with llama.cpp

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 27.3B
• Weights: about 13.7 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 921,817
• Likes: 103
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: gguf, qwen3.8-27b, imatrix, agentionai, image-text-to-text, conversational

Links

• Model: https://huggingface.co/agentionai/Qwen3.8-27B-AP-GGUF
• Files: https://huggingface.co/agentionai/Qwen3.8-27B-AP-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download agentionai/Qwen3.8-27B-AP-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.