--- title: "Qwen3.8-27B-QSA-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/84fe4bb3-66d7-4d48-a711-b5d13a08cab6" updated: "2026-11-12" --- # Qwen3.8-27B-QSA-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Qwen3.8-27B with sparse attention from GGUF files in Q8\_0 and Unsloth Q4, Q5 and Q6 mixes, with text, image and video input. For Linux users with NVIDIA GPUs. Qwen3.8-27B-QSA-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by higa2515 ## About ### What it is This repository holds GGUF copies of Qwen3.8-27B that add Qwen Sparse Attention, or QSA. You get a Q8\_0 file and three Unsloth mixed-precision files, all with the same FP32 QSA indexers. They run on a dedicated llama.cpp engine that handles text, image and video input. ### Who it suits It suits people who run large models locally on Linux with NVIDIA GPUs and want to try sparse attention on a 27B model. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers running local models | Launch a quantized model with the QSA engine | Serve the Q8\_0 file on a two-GPU Linux machine | | Research | Teams studying sparse attention | Compare QSA with the original attention design | Check attention savings on long contexts | | Vision work | People who need image and video input | Load the vision projector file with the model | Ask questions about a picture | People on other systems or GPUs can skip it for now, since the captured engine targets Linux x86\_64 and NVIDIA sm70. ### Where to use it - Linux x86\_64 with NVIDIA sm70 GPUs, which the captured engine targets - The Qwen3.8-27B QSA build of llama.cpp, since stock llama.cpp does not implement this path - Python with pip, to fetch the files with the Hugging Face tool | Use case | Who | What they get back | | --- | --- | --- | | Local chat and coding | Developers | A running local model with prefix-cache reuse | | Image and video questions | Teams with vision needs | Answers that use the vision projector file | | Quantization comparison | Researchers | Q8\_0 and Unsloth variants to test side by side | ### How to set it up You need these first: - A Linux x86\_64 machine with an NVIDIA sm70 GPU, which the captured binary targets - Python with pip, to install the Hugging Face tool - Enough GPU memory, since the card's reference setup uses Q8\_0 on two 32GB GPUs 1. Install the Hugging Face tool and pick one model file. 2. Download that file with the vision projector, the engine archive and the checksum files. 3. Check the checksums and unpack the engine archive. 4. Run the launcher with your chosen model. ```sh python3 -m pip install --user -U huggingface_hub QSA_VARIANT=Qwen3.8-27B-QSA-Q8_0.gguf hf download higa2515/Qwen3.8-27B-QSA-GGUF "$QSA_VARIANT" mmproj-Qwen3.8-27B-QSA-F16.gguf qsa27b-linux-x86_64-sm70.tar.gz SHA256SUMS release-manifest.json --local-dir qsa27b cd qsa27b sha256sum -c SHA256SUMS --ignore-missing tar -xzf qsa27b-linux-x86_64-sm70.tar.gz QSA_MODEL="$QSA_VARIANT" QSA_MODEL_DIR="$PWD" ./qsa27b-runtime/launch.sh ``` ### How to use it Run the launcher with your model, and it starts a local server on your machine. To use an Unsloth variant instead, set QSA\_VARIANT to the file name of the UD-Q6\_K, UD-Q5\_K\_M or UD-Q4\_K\_M variant, and fetch that file before you launch. No service is installed automatically. | Tool | What it does | | --- | --- | | hf download | Fetches the model, the vision projector, the engine archive and the checksum files | | sha256sum -c | Checks the downloaded files against the checksum list | | tar -xzf | Unpacks the captured engine and its launcher | | launch.sh | Starts the engine with the model you chose | ### Example You pick the Q8\_0 file, download it with the projector and engine archive, check the sums, unpack and run the launcher. The model then runs under the QSA engine on your machine, ready for text, image or video prompts as the card describes. ### For newcomers and experts GGUF is a file format for running models locally, and these files are compressed copies of a 27B model. A first small step is to read the file table, choose Q8\_0 if your GPUs have room, and follow the commands above in order. QSA uses a learned, compressed indexer to choose which context to attend to, in blocks of four tokens. The 16 full-attention layers use a fixed K of 2,048 plus the current block, and the other 48 Gated DeltaNet layers stay as they were. All KV entries stay allocated, so QSA cuts attention computation and KV reads but not KV-cache storage. The Unsloth variants reuse 866 original tensors byte for byte and add 80 FP32 indexer tensors without retraining or requantization. Choose a quantized model file → Download it with the engine archive → Check the SHA256 sums → Unpack and run the launcher → Use the local server ### Good to know - Context: the reference setup uses a context of 262,144, with a batch of 2,048 and MTP max draft 2 - Prefix reuse: the cache works in one live slot only and is lost when the server restarts - Reference path: in the BF16 Transformers reference, picture and video chats use dense attention instead of QSA - Versions: BF16 source weights are available separately File size in GB | Label | Value | | --- | --- | | Q8\_0 | 29.26 GB | | UD-Q4\_K\_M | 16.67 GB | | UD-Q5\_K\_M | 19.98 GB | | UD-Q6\_K | 22.19 GB | - License: apache-2.0 - Cost to run: none named, no API key or account. The reference setup uses two 32GB NVIDIA GPUs - Data it reaches: it downloads model files from Hugging Face, and the launcher starts a local server without installing a service - Not for: stock llama.cpp, other GPUs than sm70 without changes, or keeping the prefix cache across restarts ## README ````markdown # Qwen3.8-27B-QSA-GGUF Runs Qwen3.8-27B with sparse attention from GGUF files in Q8_0 and Unsloth Q4, Q5 and Q6 mixes, with text, image and video input. For Linux users with NVIDIA GPUs. Made by higa2515 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds GGUF copies of Qwen3.8-27B that add Qwen Sparse Attention, or QSA. You get a Q8_0 file and three Unsloth mixed-precision files, all with the same FP32 QSA indexers. They run on a dedicated llama.cpp engine that handles text, image and video input. Who it suits: It suits people who run large models locally on Linux with NVIDIA GPUs and want to try sparse attention on a 27B model. ```table Industry | Who uses it | How they use it | Example Software | Developers running local models | Launch a quantized model with the QSA engine | Serve the Q8_0 file on a two-GPU Linux machine Research | Teams studying sparse attention | Compare QSA with the original attention design | Check attention savings on long contexts Vision work | People who need image and video input | Load the vision projector file with the model | Ask questions about a picture ``` People on other systems or GPUs can skip it for now, since the captured engine targets Linux x86_64 and NVIDIA sm70. Where to use it: - Linux x86_64 with NVIDIA sm70 GPUs, which the captured engine targets - The Qwen3.8-27B QSA build of llama.cpp, since stock llama.cpp does not implement this path - Python with pip, to fetch the files with the Hugging Face tool ```table Use case | Who | What they get back Local chat and coding | Developers | A running local model with prefix-cache reuse Image and video questions | Teams with vision needs | Answers that use the vision projector file Quantization comparison | Researchers | Q8_0 and Unsloth variants to test side by side ``` How to set it up: You need these first: - A Linux x86_64 machine with an NVIDIA sm70 GPU, which the captured binary targets - Python with pip, to install the Hugging Face tool - Enough GPU memory, since the card's reference setup uses Q8_0 on two 32GB GPUs 1. Install the Hugging Face tool and pick one model file. 2. Download that file with the vision projector, the engine archive and the checksum files. 3. Check the checksums and unpack the engine archive. 4. Run the launcher with your chosen model. ```sh python3 -m pip install --user -U huggingface_hub QSA_VARIANT=Qwen3.8-27B-QSA-Q8_0.gguf hf download higa2515/Qwen3.8-27B-QSA-GGUF "$QSA_VARIANT" mmproj-Qwen3.8-27B-QSA-F16.gguf qsa27b-linux-x86_64-sm70.tar.gz SHA256SUMS release-manifest.json --local-dir qsa27b cd qsa27b sha256sum -c SHA256SUMS --ignore-missing tar -xzf qsa27b-linux-x86_64-sm70.tar.gz QSA_MODEL="$QSA_VARIANT" QSA_MODEL_DIR="$PWD" ./qsa27b-runtime/launch.sh ``` How to use it: Run the launcher with your model, and it starts a local server on your machine. To use an Unsloth variant instead, set QSA_VARIANT to the file name of the UD-Q6_K, UD-Q5_K_M or UD-Q4_K_M variant, and fetch that file before you launch. No service is installed automatically. ```table Tool | What it does hf download | Fetches the model, the vision projector, the engine archive and the checksum files sha256sum -c | Checks the downloaded files against the checksum list tar -xzf | Unpacks the captured engine and its launcher launch.sh | Starts the engine with the model you chose ``` Example: You pick the Q8_0 file, download it with the projector and engine archive, check the sums, unpack and run the launcher. The model then runs under the QSA engine on your machine, ready for text, image or video prompts as the card describes. For newcomers and experts: GGUF is a file format for running models locally, and these files are compressed copies of a 27B model. A first small step is to read the file table, choose Q8_0 if your GPUs have room, and follow the commands above in order. QSA uses a learned, compressed indexer to choose which context to attend to, in blocks of four tokens. The 16 full-attention layers use a fixed K of 2,048 plus the current block, and the other 48 Gated DeltaNet layers stay as they were. All KV entries stay allocated, so QSA cuts attention computation and KV reads but not KV-cache storage. The Unsloth variants reuse 866 original tensors byte for byte and add 80 FP32 indexer tensors without retraining or requantization. ```flow Choose a quantized model file Download it with the engine archive Check the SHA256 sums Unpack and run the launcher Use the local server ``` Good to know: - Context: the reference setup uses a context of 262,144, with a batch of 2,048 and MTP max draft 2 - Prefix reuse: the cache works in one live slot only and is lost when the server restarts - Reference path: in the BF16 Transformers reference, picture and video chats use dense attention instead of QSA - Versions: BF16 source weights are available separately ```chart Title: File size in GB Q8_0: 29.26 GB UD-Q4_K_M: 16.67 GB UD-Q5_K_M: 19.98 GB UD-Q6_K: 22.19 GB ``` - License: apache-2.0 - Cost to run: none named, no API key or account. The reference setup uses two 32GB NVIDIA GPUs - Data it reaches: it downloads model files from Hugging Face, and the launcher starts a local server without installing a service - Not for: stock llama.cpp, other GPUs than sm70 without changes, or keeping the prefix cache across restarts ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: gguf - Parameters: 27.4B - Weights: about 13.7 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 12,136 - Likes: 32 - License: apache-2.0 - Based on: higa2515/Qwen3.8-27B-QSA-BF16 - Tags: gguf, multimodal, llama-cpp, q8_0, q4_k_m, q5_k_m, q6_k, unsloth, image-text-to-text, conversational ## Links - Model: https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF - Files: https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download higa2515/Qwen3.8-27B-QSA-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [higa2515/Qwen3.8-27B-QSA-GGUF on Hugging Face](https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-QSA-GGUF (this page) | 12,136 | 32 | 27.4B | apache-2.0 | 2026-11-05 | | [Qwen3.5-0.8B-weight-space-programming-GGUF by NotCherry](https://mindket.com/p/ab2cda48-8af8-4517-8b27-ea68db4495c5) | 1,731 | 3 | 773M | apache-2.0 | 2026-11-03 | | [Xiaomi-OCR-0-GGUF](https://mindket.com/p/1921d172-02df-42f8-b8be-4fa52ab31210) | 44,340 | 19 | 752M | apache-2.0 | 2026-10-30 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [Qwen2.5-VL-7B-Instruct-GGUF](https://mindket.com/p/9cac821c-b890-4e97-b7ad-543fcae42d3b) | 285,067 | 253 | 7.6B | apache-2.0 | 2025-05-12 | | [Qwopus3.8-27B-Flash-GGUF](https://mindket.com/p/509fda01-c99c-450a-8d3e-2444ec0c6af7) | 2,153 | 248 | 461M | apache-2.0 | 2026-11-07 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-05 - First published: 2026-11-04 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open higa2515/Qwen3.8-27B-QSA-GGUF on Hugging Face](https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/84fe4bb3-66d7-4d48-a711-b5d13a08cab6) - [Its page on Mindket](https://mindket.com/p/84fe4bb3-66d7-4d48-a711-b5d13a08cab6)