--- title: "Qwen3.8-27B-QSA-GSQ-RCO-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/ce87f21f-5958-44f5-a952-a31b4dc6e24e" updated: "2026-11-13" --- # Qwen3.8-27B-QSA-GSQ-RCO-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Qwen3.8-27B with sparse attention on a single 24 GB GPU at a 262K-token context with image input, as GGUF files for people with a custom llama.cpp engine build. Qwen3.8-27B-QSA-GSQ-RCO-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by pentacoxian-dev ## About ### What it is The files here let you run Qwen3.8-27B with Qwen's sparse attention design on a single 24 GB graphics card. It handles a context of 262K tokens and accepts images. The files need a special build of llama.cpp from pentaCoxian, because a standard llama.cpp build will not load these weights. ### Who it suits It suits advanced users with a 24 GB GPU who want to read long documents or large codebases with a 27B model on their own machine. Who uses this model | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Query a large codebase locally | Ask where a function is defined in a large repository | | Research | Local-model enthusiasts | Test sparse attention at long context | Compare sparse and dense attention recall | | AI infrastructure | Self-hosters | Serve a 27B model on one consumer GPU | Run the serve script on an RTX 4090 | If you cannot build the custom engine or your GPU has less than 24 GB, you can skip it. ### Where to use it - The Qwen-QSA-llama.cpp engine built from its source - A machine with one 24 GB GPU - Windows, macOS or Linux with Python, for downloading the files Jobs it does | Use case | Who | What they get back | | --- | --- | --- | | Long-context questions | Developers | Answers drawn from inputs up to 262K tokens | | Image questions | Anyone with pictures to check | Text answers about an image you supply | | Faster decoding | Self-hosters | Draft-assisted generation with the bundled drafter | ### How to set it up You need a built checkout of the engine, and its README has the build steps. The three files total about 14 GB. - Python with pip, for the hf command - A built copy of the Qwen-QSA-llama.cpp engine - One 24 GB GPU 1. Build the engine by following its README. 2. Download the model files into a models folder. 3. Run the serve script from the engine checkout. ```sh pip install -U huggingface_hub ``` ```sh hf download pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF --local-dir models deploy/qsa27b-24gb/serve.sh ``` ### How to use it Run the two commands from inside the built engine checkout. The script serves the main model, the drafter and the vision projector together. A SHA256SUMS file lists checksums so you can verify the downloads. Files in the repository | Tool | What it does | | --- | --- | | Main IQ3\_S model file | The quantized 27B weights, with the sparse-attention indexers and an MTP head built in, 12.3 GB | | Qwen3.8-27B-DFlash2-IQ4\_XS.gguf | Drafter for speculative decoding, 1.0 GB | | mmproj-Qwen3.8-27B-QSA-F16.gguf | Vision projector, 0.9 GB | ### Example You start the server, then give it a long source file and ask for the definition of one function. The model answers in text from its sparse attention, and the card's recall test reports 67% of exact lookups fully correct at 255K tokens. ### For newcomers and experts This is an advanced setup. Normal attention looks at every earlier token, while sparse attention looks at a chosen few, which saves speed and memory on long inputs. A first small step is to read the engine README before downloading anything. In each of its 16 attention layers, a small indexer chooses 2,048 tokens for the model to look at. The original indexers were trained at 16K, so at 255K their picks fell behind. The authors retrained only the indexers, about 53M parameters, against the model's own dense attention. Turning sparse attention off with LLAMA\_QSA\_DENSE=1 shows dense results at about half the decode speed. Qwen3.8-27B IQ3\_S base file → QSA indexers added and retrained → Drafter and vision projector alongside → Served by the custom engine ### Good to know On one RTX 4090 at 262K context the author measured about 114.7 tokens per second for greedy decoding and a peak of 22.8 GiB of video memory. Beyond 64K, sparse attention still misses some lookups, most often when the target record is in the middle of the text. Exact-lookup recall at 255K context | Label | Value | | --- | --- | | Previous indexers | 41% | | Retrained indexers | 67% | | Dense attention | 92% | - License: Apache 2.0. - Cost to run: none named, apart from a 24 GB GPU. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: stock llama.cpp, which cannot run these sparse-attention weights. ## README ````markdown # Qwen3.8-27B-QSA-GSQ-RCO-GGUF Runs Qwen3.8-27B with sparse attention on a single 24 GB GPU at a 262K-token context with image input, as GGUF files for people with a custom llama.cpp engine build. Made by pentacoxian-dev on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: The files here let you run Qwen3.8-27B with Qwen's sparse attention design on a single 24 GB graphics card. It handles a context of 262K tokens and accepts images. The files need a special build of llama.cpp from pentaCoxian, because a standard llama.cpp build will not load these weights. Who it suits: It suits advanced users with a 24 GB GPU who want to read long documents or large codebases with a 27B model on their own machine. ```table Title: Who uses this model Industry | Who uses it | How they use it | Example Software | Developers | Query a large codebase locally | Ask where a function is defined in a large repository Research | Local-model enthusiasts | Test sparse attention at long context | Compare sparse and dense attention recall AI infrastructure | Self-hosters | Serve a 27B model on one consumer GPU | Run the serve script on an RTX 4090 ``` If you cannot build the custom engine or your GPU has less than 24 GB, you can skip it. Where to use it: - The Qwen-QSA-llama.cpp engine built from its source - A machine with one 24 GB GPU - Windows, macOS or Linux with Python, for downloading the files ```table Title: Jobs it does Use case | Who | What they get back Long-context questions | Developers | Answers drawn from inputs up to 262K tokens Image questions | Anyone with pictures to check | Text answers about an image you supply Faster decoding | Self-hosters | Draft-assisted generation with the bundled drafter ``` How to set it up: You need a built checkout of the engine, and its README has the build steps. The three files total about 14 GB. - Python with pip, for the hf command - A built copy of the Qwen-QSA-llama.cpp engine - One 24 GB GPU 1. Build the engine by following its README. 2. Download the model files into a models folder. 3. Run the serve script from the engine checkout. ```sh pip install -U huggingface_hub ``` ```sh hf download pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF --local-dir models deploy/qsa27b-24gb/serve.sh ``` How to use it: Run the two commands from inside the built engine checkout. The script serves the main model, the drafter and the vision projector together. A SHA256SUMS file lists checksums so you can verify the downloads. ```table Title: Files in the repository Tool | What it does Main IQ3_S model file | The quantized 27B weights, with the sparse-attention indexers and an MTP head built in, 12.3 GB Qwen3.8-27B-DFlash2-IQ4_XS.gguf | Drafter for speculative decoding, 1.0 GB mmproj-Qwen3.8-27B-QSA-F16.gguf | Vision projector, 0.9 GB ``` Example: You start the server, then give it a long source file and ask for the definition of one function. The model answers in text from its sparse attention, and the card's recall test reports 67% of exact lookups fully correct at 255K tokens. For newcomers and experts: This is an advanced setup. Normal attention looks at every earlier token, while sparse attention looks at a chosen few, which saves speed and memory on long inputs. A first small step is to read the engine README before downloading anything. In each of its 16 attention layers, a small indexer chooses 2,048 tokens for the model to look at. The original indexers were trained at 16K, so at 255K their picks fell behind. The authors retrained only the indexers, about 53M parameters, against the model's own dense attention. Turning sparse attention off with LLAMA_QSA_DENSE=1 shows dense results at about half the decode speed. ```flow Qwen3.8-27B IQ3_S base file QSA indexers added and retrained Drafter and vision projector alongside Served by the custom engine ``` Good to know: On one RTX 4090 at 262K context the author measured about 114.7 tokens per second for greedy decoding and a peak of 22.8 GiB of video memory. Beyond 64K, sparse attention still misses some lookups, most often when the target record is in the middle of the text. ```chart Title: Exact-lookup recall at 255K context Previous indexers: 41% Retrained indexers: 67% Dense attention: 92% ``` - License: Apache 2.0. - Cost to run: none named, apart from a 24 GB GPU. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: stock llama.cpp, which cannot run these sparse-attention weights. ## On Hugging Face, 13 November 2026 - Parameters: 1.9B - Weights: about 962 MB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 2,784 - Likes: 6 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Tags: gguf, llama.cpp, gsq-rco, conversational ## Links - Model: https://huggingface.co/pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF - Files: https://huggingface.co/pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-QSA-GSQ-RCO-GGUF (this page) | 2,784 | 6 | 1.9B | apache-2.0 | 2026-11-08 | | [SenseNova-6.8-Code2Video GSQ-RCO GGUF](https://mindket.com/p/78f43f93-2a33-400e-a778-71e873414af0) | 13,778 | 9 | 26.9B | apache-2.0 | 2026-10-29 | | [Qwen3.8-35B-A3B-Distill-GGUF](https://mindket.com/p/d707b0c8-89b8-4c4f-8d49-2ff67de613c7) | 1,337,097 | 277 | 35.5B | apache-2.0 | 2026-09-16 | | [Tess-4.5-27B GGUF](https://mindket.com/p/2c9f4a01-ed9d-4dd7-94a5-94a5fe30c6a8) | 1,095,620 | 65 | 26.9B | apache-2.0 | 2026-11-08 | | [Sharp-Spark-X2.5-4B-GGUF](https://mindket.com/p/d4d90f81-4886-4de3-ab95-d64c344a708c) | 875,346 | 164 | 4.1B | apache-2.0 | 2026-11-04 | | [Qwen3.8-4B-Distill-GGUF](https://mindket.com/p/ef3614cf-441f-4388-b81f-7b72b3e7ed30) | 570,974 | 208 | 4.3B | apache-2.0 | 2026-08-16 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-08 - First published: 2026-11-04 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/pentacoxian-dev/Qwen3.8-27B-QSA-GSQ-RCO-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/ce87f21f-5958-44f5-a952-a31b4dc6e24e) - [Its page on Mindket](https://mindket.com/p/ce87f21f-5958-44f5-a952-a31b4dc6e24e)