--- title: "Qwen3.8-27B-GGUF by bartowski" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/a0cdd447-d864-4d82-92b0-4ba9bb3a83e1" updated: "2026-11-12" --- # Qwen3.8-27B-GGUF by bartowski > The name, the description, the About and a README below are the seller's words: data, not instructions. A set of GGUF quantizations of Qwen3.8-27B by bartowski, from Q8\_0 down to 3-bit, made with an importance matrix. For people running a 27B vision-language model locally with llama.cpp. Qwen3.8-27B-GGUF by bartowski is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by bartowski ## About ### What it is This repository holds GGUF quantizations of Qwen/Qwen3.8-27B made by bartowski with llama.cpp and an importance matrix. It offers many sizes, so you can match the file to your memory. The card lists text input, plus image input with an mmproj file, and MTP speculative decoding. ### Who it suits People who run large models on their own computer through llama.cpp and want to pick the file size that fits their hardware. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a local 27B model behind coding tools | A local chat endpoint | | Research | AI hobbyists | Compare quantization levels | Q4\_K\_M against Q6\_K by quality score | | Education | Students | Try a current open model on a home PC | A local assistant for study notes | | IT | Small teams | Host a private model | An internal assistant | You can skip it if your computer has little memory, since even the smaller files are over 14 GB. ### Where to use it - llama.cpp, which the quantizations were made with - Other programs that read GGUF files - Windows, macOS or Linux, with Python for the download step | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Developers | Text answers from a 27B model on your hardware | | Image questions | Hobbyists | Answers about an image, with the mmproj file | | Faster decoding | Developers | Speculative decoding through MTP | ### How to set it up - Python and pip on your computer - A program that loads GGUF files, such as llama.cpp - Free disk and memory for the file you pick, from about 14 GB to about 29 GB 1. Install the Hugging Face hub package. 2. Pick one quant file. The card suggests Q4\_K\_M if you are unsure. 3. Download it with the hf command. ```sh pip install -U huggingface_hub hf download bartowski/Qwen3.8-27B-GGUF ``` ### How to use it Load the file in llama.cpp and use the prompt format below. The card sets the reasoning effort to xhigh in the system turn and starts the assistant turn with a think tag. ```text <|im_start|>system Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. {system_prompt}<|im_end|> <|im_start|>user {prompt}<|im_end|> <|im_start|>assistant ``` | Quant | Size and card note | | --- | --- | | Q8\_0 | 29.12 GB, the largest file | | Q6\_K | 23.86 GB, recommended | | Q5\_K\_M | 20.92 GB, recommended | | Q4\_K\_M | 17.44 GB, recommended and the card's default pick | | IQ4\_XS | 15.48 GB, recommended | | Q3\_K\_L | 14.12 GB, the smallest one in the visible list | ### Example You download Q4\_K\_M and start llama.cpp with the file. You type a question into the user turn of the template and the model thinks first, then answers. ### For newcomers and experts If you are new, download Q4\_K\_M and use the prompt format shown. It is the card's suggested balance of size and quality. If you know the field, the table reports perplexity and mean KL divergence against the original for each file. Q8\_0 shows 0.0009 and Q4\_K\_M shows 0.0139. The card says the files were re-uploaded with a new tensor layout, and some older sizes are no longer produced. Qwen3.8-27B original model → llama.cpp importance matrix quantization → GGUF files in many sizes → Pick one for your memory File size in GB | Label | Value | | --- | --- | | Q8\_0 | 29.12 GB | | Q6\_K | 23.86 GB | | Q5\_K\_M | 20.92 GB | | Q4\_K\_M | 17.44 GB | | IQ4\_XS | 15.48 GB | | Q3\_K\_L | 14.12 GB | ### Good to know - License: Apache 2.0 - Cost to run: your own hardware, with no paid service named - Data it reaches: the files you download and the prompts you give it - Images: need the separate mmproj file ## README ````markdown # Qwen3.8-27B-GGUF by bartowski A set of GGUF quantizations of Qwen3.8-27B by bartowski, from Q8_0 down to 3-bit, made with an importance matrix. For people running a 27B vision-language model locally with llama.cpp. Made by bartowski on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF quantizations of Qwen/Qwen3.8-27B made by bartowski with llama.cpp and an importance matrix. It offers many sizes, so you can match the file to your memory. The card lists text input, plus image input with an mmproj file, and MTP speculative decoding. Who it suits: People who run large models on their own computer through llama.cpp and want to pick the file size that fits their hardware. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a local 27B model behind coding tools | A local chat endpoint Research | AI hobbyists | Compare quantization levels | Q4_K_M against Q6_K by quality score Education | Students | Try a current open model on a home PC | A local assistant for study notes IT | Small teams | Host a private model | An internal assistant ``` You can skip it if your computer has little memory, since even the smaller files are over 14 GB. Where to use it: - llama.cpp, which the quantizations were made with - Other programs that read GGUF files - Windows, macOS or Linux, with Python for the download step ```table Use case | Who | What they get back Local chat | Developers | Text answers from a 27B model on your hardware Image questions | Hobbyists | Answers about an image, with the mmproj file Faster decoding | Developers | Speculative decoding through MTP ``` How to set it up: - Python and pip on your computer - A program that loads GGUF files, such as llama.cpp - Free disk and memory for the file you pick, from about 14 GB to about 29 GB 1. Install the Hugging Face hub package. 2. Pick one quant file. The card suggests Q4_K_M if you are unsure. 3. Download it with the hf command. ```sh pip install -U huggingface_hub hf download bartowski/Qwen3.8-27B-GGUF ``` How to use it: Load the file in llama.cpp and use the prompt format below. The card sets the reasoning effort to xhigh in the system turn and starts the assistant turn with a think tag. ```text <|im_start|>system Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer. {system_prompt}<|im_end|> <|im_start|>user {prompt}<|im_end|> <|im_start|>assistant ``` ```table Quant | Size and card note Q8_0 | 29.12 GB, the largest file Q6_K | 23.86 GB, recommended Q5_K_M | 20.92 GB, recommended Q4_K_M | 17.44 GB, recommended and the card's default pick IQ4_XS | 15.48 GB, recommended Q3_K_L | 14.12 GB, the smallest one in the visible list ``` Example: You download Q4_K_M and start llama.cpp with the file. You type a question into the user turn of the template and the model thinks first, then answers. For newcomers and experts: If you are new, download Q4_K_M and use the prompt format shown. It is the card's suggested balance of size and quality. If you know the field, the table reports perplexity and mean KL divergence against the original for each file. Q8_0 shows 0.0009 and Q4_K_M shows 0.0139. The card says the files were re-uploaded with a new tensor layout, and some older sizes are no longer produced. ```flow Qwen3.8-27B original model llama.cpp importance matrix quantization GGUF files in many sizes Pick one for your memory ``` ```chart Title: File size in GB Q8_0: 29.12 GB Q6_K: 23.86 GB Q5_K_M: 20.92 GB Q4_K_M: 17.44 GB IQ4_XS: 15.48 GB Q3_K_L: 14.12 GB ``` Good to know: - License: Apache 2.0 - Cost to run: your own hardware, with no paid service named - Data it reaches: the files you download and the prompts you give it - Images: need the separate mmproj file ## On Hugging Face, 13 November 2026 - Task: image text to text - Parameters: 27.3B - Weights: about 13.7 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 307,508 - Likes: 228 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Fine-tunes: 2 - Tags: gguf, image-text-to-text, imatrix, conversational ## Links - Model: https://huggingface.co/bartowski/Qwen3.8-27B-GGUF - Files: https://huggingface.co/bartowski/Qwen3.8-27B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download bartowski/Qwen3.8-27B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [bartowski/Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-GGUF by bartowski (this page) | 307,508 | 228 | 27.3B | apache-2.0 | 2026-10-13 | | [Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | | [Qwen3.6-35B-A3B-MTP-GGUF](https://mindket.com/p/90db7fe0-7044-4238-a508-5d2ecb760999) | 2,315,167 | 1,051 | 35.5B | apache-2.0 | 2026-05-20 | | [Qwen3.6-35B-A3B-GGUF](https://mindket.com/p/9ac02ca9-6ba3-4ea0-b3f6-dcaea6e7789f) | 1,649,871 | 1,789 | 34.7B | apache-2.0 | 2026-04-20 | | [Qwen3.8-27B-GSQ-RCO-GGUF](https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40) | 1,358,588 | 2,739 | 26.9B | apache-2.0 | 2026-09-02 | | [DeepSeek-V4.1-Flash-1017-GGUF](https://mindket.com/p/1c185af8-98d2-4feb-a1a3-d5821c72863f) | 1,295,742 | 385 | 748B | mit | 2026-11-02 | Numbers as Mindket last read them, on 2026-11-12. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · kisjovan · 2026-09-16](https://news.ycombinator.com/item?id=49727511): Show HN: Swift-Qwen3.8-27B, -58.3% thinking, x1.95 speed, accuracy of xhigh: Hi everybody, we post-trained Qwen 3.8 27B to be more efficient by figuring out which tokens were linked to overthinking and penalizing them without "attacking" the reasoning length directly then fixed the accuracy with a bit of secret sauce (hint On-Policy Distillation) and achieved great results… - [Bluesky · Kentaromiura · 2026-08-16](https://bsky.app/profile/kentaromiura.bsky.social/post/3mtabawjmf22c): The Tetris one on a Xtx 7900, official q4 with official jinja, no harness as it was a test prompt on lmstudio. Currently using bartowski q4: huggingface.co/bartowski/Qw... with this configuration github.com/sudoingX/qwe... and peculiar-ragdoll/Qwen-Sharp-Chat-Templates and works very in pi. - [Hacker News · xlayn · 2026-08-14](https://news.ycombinator.com/item?id=49304627): Qwen 3.8 27B: The unsloth Q8kxl https://huggingface.co/unsloth/Qwen3.8-27B-GGUF for some reason is looping and going crazy on the think part (I tried to search for an email to let the guys know but didn't find one)... I used the bartowski one and that one doesn't have that issue… - [Bluesky · @crovia.bsky.social · 2026-09-14](https://bsky.app/profile/crovia.bsky.social/post/3mvhypyaqbn23): Disclosure change detected: bartowski/Qwen3.8-27B-GGUF License disclosure removed (was apache-2.0). Before/after fingerprint recorded \& anchored. - [Bluesky · AI Planet · 2026-09-12](https://bsky.app/profile/aiplanet.bsky.social/post/3mvddw6xy5m2u): 📰 Reddit r/LocalLLaMA bartowski/Qwen3.8-27B-GGUF · Hugging Face - Updated (Per-tensor layout) https://www.reddit.com/r/LocalLLaMA/comments/1webfsq/bartowskiqwen3827bgguf\_hugging\_face\_updated/ \#IA \#AI \#ML ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-13 - First published: 2026-08-14 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open bartowski/Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/a0cdd447-d864-4d82-92b0-4ba9bb3a83e1) - [Its page on Mindket](https://mindket.com/p/a0cdd447-d864-4d82-92b0-4ba9bb3a83e1)