Qwen3.8-27B-GSQ-RCO-GGUF

Qwen3.8-27B-GSQ-RCO-GGUF

ISTA-DASLab's GGUF versions of Qwen3.8-27B in four sizes, each tensor quantized at its own precision, with a vision projector for images. Suits people who run the model locally in llama.cpp, Ollama or LM Studio.

Made by ISTA-DASLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds four GGUF versions of Qwen3.8-27B from ISTA-DASLab, plus the model's vision projector file for image use. Rather than one precision for every weight, each tensor gets its own quantization type, chosen by a gradient based search within a size budget. The methods are called GSQ and RCO. The files are ordinary GGUF.

Who it suits: People who run a 27B vision language model on their own computer and want a choice of file size, and researchers who study quantization.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a 27B model locally in a smaller file | Load the IQ3_S file in llama.cpp
Research | Quantization researchers | Study how bits are spread across tensors | Compare the four file sizes and bit widths
Hobby computing | Local model users | Chat with images on a home computer | Load the vision projector with the model

Skip it if you need the full precision model, because these are compressed versions.

Where to use it:

• llama.cpp
• Ollama
• LM Studio

Use case | Who | What they get back
Local chat | Anyone | Answers from a 27B model in a file of 8.4 to 11.8 GB
Image questions | Local model users | Answers about pictures, using the vision projector file
Quantization study | Researchers | Per tensor quantization types to examine

How to set it up:

• Python, for the Hugging Face tools
• Free disk space for one of the four files, 8.4 to 11.8 GB, plus the vision projector file
• One of the programs listed above

1. Install the Hugging Face hub package.
2. Download the repository, or just the file size you want.
3. Load the GGUF file, and the vision projector for images, in your program.

pip install -U huggingface_hub
hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

How to use it: Pick a size, load that file in your program and chat. Add the mmproj file when you want to send images. One copy of it serves every size.

Tool | What it does
IQ2_XS file | Smallest size, 2.50 bits per weight and 8.4 GB
IQ2_S file | 2.75 bits and 9.3 GB, said to match the base model on one maths test
IQ3_XXS file | 3.00 bits and 10.1 GB, an all round choice
IQ3_S file | 3.50 bits and 11.8 GB, the size the card recommends
mmproj file | Holds the vision encoder and projector at BF16

Example: Load the IQ3_S file and the mmproj file in LM Studio, attach a photo and ask what it shows. The model should answer with a written description of the picture.

For newcomers and experts: If you are new to this, start with the recommended IQ3_S file and ask a plain text question before you try an image.

If you know the field, GSQ learns grid assignments and group scales with a Gumbel softmax relaxation. RCO assigns one of several quantization types to each tensor under a total size budget, using a Riemannian reformulation of the constraint.

GSQ quantizes each tensor at a given type
RCO assigns types under a size budget
The result is saved as GGUF files
Your program loads one file

Title: File size by quantization class
IQ2_XS: 8.4 GB
IQ2_S: 9.3 GB
IQ3_XXS: 10.1 GB
IQ3_S: 11.8 GB

Good to know:

• The card's quality claims, such as task lossless at 3.50 bits, are the publisher's own.
• The card also has a plot on speculative decoding with the MTP head.
• The base model is Qwen/Qwen3.8-27B.
• License: apache-2.0.
• Cost to run: none named, only your own hardware.
• Data it reaches: the prompts and images you give it. The source names no account or outside service.
• Not for: the source names none.

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 26.9B
• Weights: about 13.4 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 1,358,588
• Likes: 2,739
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 8
• Fine-tunes: 4
• Tags: gguf, quantization, mixed-precision, ist-daslab, multimodal, vision, image-text-to-text, imatrix, conversational

Links

• Model: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
• Files: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.