Qwen3.8-27B-SLQ-GGUF

Qwen3.8-27B-SLQ-GGUF

Compresses Qwen3.8-27B into three GGUF files with different size and fidelity levels, using a custom statistically lossless quantization. For people who run the model locally in llama.cpp.

Made by kaitchup on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds Qwen3.8-27B compressed with a custom version of SLQ, short for Statistically-Lossless Quantization, and packed as GGUF. Instead of one uniform precision, the search picks a source for each of 498 whole matrices, including the embeddings and the output head. You choose one of three files.

Who it suits: It suits people who run large models locally and want to pick a precise point between file size and closeness to the original.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a 27B model locally in llama.cpp | A local coding or chat assistant
Research | Quantization researchers | Study per-matrix mixed precision and EAR scores | Compare the three files on your own prompts
AI hobbyists | Local model users | Choose a file that fits their memory | Pick the smallest file when memory is tight
Operations | Platform teams | Serve a faithful compressed copy of the base model | An internal text endpoint

If you need image or video input, you can skip it, because no vision tensors or projector are included.

Where to use it:

• llama.cpp builds with Qwen3.8 architecture support
• Windows, macOS or Linux, where you download the files with Python
• Other programs that read GGUF files

Use case | Who | What they get back
Text chat | Local model users | Replies from a close copy of Qwen3.8-27B
Size choice | Developers | Three files, from 18.876 GB to 23.697 GB
Fidelity study | Researchers | Measured EAR@10 values near 98, 98.5 and 99 percent

How to set it up: You need Python with pip and a llama.cpp build that supports Qwen3.8.

• Python and pip
• A llama.cpp build with Qwen3.8 support
• Disk space for the file you choose, from 18.876 GB to 23.697 GB

1. Install the Hugging Face client with pip.
2. Download one file from the repository, not all three.
3. Load that file in llama.cpp.

pip install -U huggingface_hub
hf download kaitchup/Qwen3.8-27B-SLQ-GGUF \
  Qwen3.8-27B-SLQ-EAR98-Q5_K_S.gguf --local-dir .

How to use it: Load the file you downloaded in a llama.cpp build and chat in text. The card validated text scoring and standalone GGUF parity, but not image or video input and not speculative decoding.

Model | Quant label | Complete file in GB | EAR@10 target
EAR98 | Q5_K_S | 18.876 | 98%
EAR98.5 | Q5_K_M | 20.703 | 98.5%
EAR99 | Q6_K | 23.697 | 99%

Example: You download the EAR98 file, load it in llama.cpp and ask a text question. You get an ordinary text reply. The card says the output stays close to the BF16 model's next-token probabilities, though it does not promise identical answers.

For newcomers and experts: Pick the file by how much memory you have. A bigger file stays closer to the original model. The first small thing to try is the smallest file, with the download command above.

EAR means Expected Acceptance Rate, the overlap between the BF16 model's next-token probabilities and the compared model's, over the top 10 likely tokens. An EAR of 0.99 is a 99% average overlap in probabilities, which is not the same as 99% accuracy on benchmarks. The filename suffixes are only Hub compatibility labels for the custom allocations. The search drew from several donor files, including unsloth GGUF types and a GSQ file.

Qwen3.8-27B BF16 reference
Candidate donor files from several quant types
Search picks a source per matrix
Custom SLQ allocation reaches an EAR target
Packed as GGUF

Good to know: Sizes: file sizes are in decimal GB. Each file keeps about 351 million bytes of MTP tensors, which were disabled during evaluation.

Title: Complete file size in GB
EAR98: 18.876 GB
EAR98.5: 20.703 GB
EAR99: 23.697 GB

• Fidelity: the values are calibration fidelity, since the same corpus guided the choice, and the card says downstream task equivalence has not been established.
• Missing parts: the files leave out vision tensors and a multimodal projector.
• Credit: the card thanks the Qwen team, Unsloth, ISTA-DASLab and the SLQ authors.
• License: Apache 2.0, as the License line says.
• Cost to run: no API key, account or paid service is named. The author invites optional support.
• Data it reaches: the prompts you give your runner, and Hugging Face for the download.
• Not for: image or video input, and speculative decoding, which the card says it did not validate.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 27.3B
• Weights: about 13.7 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 13,478
• Likes: 60
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: gguf, qwen3_5, mixed-precision, llama.cpp, quantization, text-generation, conversational

Links

• Model: https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF
• Files: https://huggingface.co/kaitchup/Qwen3.8-27B-SLQ-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download kaitchup/Qwen3.8-27B-SLQ-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.