Qwen3.8-27B-QSA-GGUF

Qwen3.8-27B-QSA-GGUF

Runs Qwen3.8-27B with sparse attention from GGUF files in Q8_0 and Unsloth Q4, Q5 and Q6 mixes, with text, image and video input. For Linux users with NVIDIA GPUs.

Made by higa2515 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF copies of Qwen3.8-27B that add Qwen Sparse Attention, or QSA. You get a Q8_0 file and three Unsloth mixed-precision files, all with the same FP32 QSA indexers. They run on a dedicated llama.cpp engine that handles text, image and video input.

Who it suits: It suits people who run large models locally on Linux with NVIDIA GPUs and want to try sparse attention on a 27B model.

Industry | Who uses it | How they use it | Example
Software | Developers running local models | Launch a quantized model with the QSA engine | Serve the Q8_0 file on a two-GPU Linux machine
Research | Teams studying sparse attention | Compare QSA with the original attention design | Check attention savings on long contexts
Vision work | People who need image and video input | Load the vision projector file with the model | Ask questions about a picture

People on other systems or GPUs can skip it for now, since the captured engine targets Linux x86_64 and NVIDIA sm70.

Where to use it:

• Linux x86_64 with NVIDIA sm70 GPUs, which the captured engine targets
• The Qwen3.8-27B QSA build of llama.cpp, since stock llama.cpp does not implement this path
• Python with pip, to fetch the files with the Hugging Face tool

Use case | Who | What they get back
Local chat and coding | Developers | A running local model with prefix-cache reuse
Image and video questions | Teams with vision needs | Answers that use the vision projector file
Quantization comparison | Researchers | Q8_0 and Unsloth variants to test side by side

How to set it up: You need these first:

• A Linux x86_64 machine with an NVIDIA sm70 GPU, which the captured binary targets
• Python with pip, to install the Hugging Face tool
• Enough GPU memory, since the card's reference setup uses Q8_0 on two 32GB GPUs

1. Install the Hugging Face tool and pick one model file.
2. Download that file with the vision projector, the engine archive and the checksum files.
3. Check the checksums and unpack the engine archive.
4. Run the launcher with your chosen model.

python3 -m pip install --user -U huggingface_hub
QSA_VARIANT=Qwen3.8-27B-QSA-Q8_0.gguf
hf download higa2515/Qwen3.8-27B-QSA-GGUF   "$QSA_VARIANT"   mmproj-Qwen3.8-27B-QSA-F16.gguf   qsa27b-linux-x86_64-sm70.tar.gz   SHA256SUMS release-manifest.json   --local-dir qsa27b
cd qsa27b
sha256sum -c SHA256SUMS --ignore-missing
tar -xzf qsa27b-linux-x86_64-sm70.tar.gz
QSA_MODEL="$QSA_VARIANT" QSA_MODEL_DIR="$PWD" ./qsa27b-runtime/launch.sh

How to use it: Run the launcher with your model, and it starts a local server on your machine. To use an Unsloth variant instead, set QSA_VARIANT to the file name of the UD-Q6_K, UD-Q5_K_M or UD-Q4_K_M variant, and fetch that file before you launch. No service is installed automatically.

Tool | What it does
hf download | Fetches the model, the vision projector, the engine archive and the checksum files
sha256sum -c | Checks the downloaded files against the checksum list
tar -xzf | Unpacks the captured engine and its launcher
launch.sh | Starts the engine with the model you chose

Example: You pick the Q8_0 file, download it with the projector and engine archive, check the sums, unpack and run the launcher. The model then runs under the QSA engine on your machine, ready for text, image or video prompts as the card describes.

For newcomers and experts: GGUF is a file format for running models locally, and these files are compressed copies of a 27B model. A first small step is to read the file table, choose Q8_0 if your GPUs have room, and follow the commands above in order.

QSA uses a learned, compressed indexer to choose which context to attend to, in blocks of four tokens. The 16 full-attention layers use a fixed K of 2,048 plus the current block, and the other 48 Gated DeltaNet layers stay as they were. All KV entries stay allocated, so QSA cuts attention computation and KV reads but not KV-cache storage. The Unsloth variants reuse 866 original tensors byte for byte and add 80 FP32 indexer tensors without retraining or requantization.

Choose a quantized model file
Download it with the engine archive
Check the SHA256 sums
Unpack and run the launcher
Use the local server

Good to know:

• Context: the reference setup uses a context of 262,144, with a batch of 2,048 and MTP max draft 2
• Prefix reuse: the cache works in one live slot only and is lost when the server restarts
• Reference path: in the BF16 Transformers reference, picture and video chats use dense attention instead of QSA
• Versions: BF16 source weights are available separately

Title: File size in GB
Q8_0: 29.26 GB
UD-Q4_K_M: 16.67 GB
UD-Q5_K_M: 19.98 GB
UD-Q6_K: 22.19 GB

• License: apache-2.0
• Cost to run: none named, no API key or account. The reference setup uses two 32GB NVIDIA GPUs
• Data it reaches: it downloads model files from Hugging Face, and the launcher starts a local server without installing a service
• Not for: stock llama.cpp, other GPUs than sm70 without changes, or keeping the prefix cache across restarts

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 27.4B
• Weights: about 13.7 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 12,136
• Likes: 32
• License: apache-2.0
• Based on: higa2515/Qwen3.8-27B-QSA-BF16
• Tags: gguf, multimodal, llama-cpp, q8_0, q4_k_m, q5_k_m, q6_k, unsloth, image-text-to-text, conversational

Links

• Model: https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF
• Files: https://huggingface.co/higa2515/Qwen3.8-27B-QSA-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download higa2515/Qwen3.8-27B-QSA-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.