--- title: "Qwen3.8-27B-GSQ-RCO-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40" updated: "2026-11-12" --- # Qwen3.8-27B-GSQ-RCO-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. ISTA-DASLab's GGUF versions of Qwen3.8-27B in four sizes, each tensor quantized at its own precision, with a vision projector for images. Suits people who run the model locally in llama.cpp, Ollama or LM Studio. Qwen3.8-27B-GSQ-RCO-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by ISTA-DASLab ## About ### What it is This repository holds four GGUF versions of Qwen3.8-27B from ISTA-DASLab, plus the model's vision projector file for image use. Rather than one precision for every weight, each tensor gets its own quantization type, chosen by a gradient based search within a size budget. The methods are called GSQ and RCO. The files are ordinary GGUF. ### Who it suits People who run a 27B vision language model on their own computer and want a choice of file size, and researchers who study quantization. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Run a 27B model locally in a smaller file | Load the IQ3\_S file in llama.cpp | | Research | Quantization researchers | Study how bits are spread across tensors | Compare the four file sizes and bit widths | | Hobby computing | Local model users | Chat with images on a home computer | Load the vision projector with the model | Skip it if you need the full precision model, because these are compressed versions. ### Where to use it - llama.cpp - Ollama - LM Studio | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Anyone | Answers from a 27B model in a file of 8.4 to 11.8 GB | | Image questions | Local model users | Answers about pictures, using the vision projector file | | Quantization study | Researchers | Per tensor quantization types to examine | ### How to set it up - Python, for the Hugging Face tools - Free disk space for one of the four files, 8.4 to 11.8 GB, plus the vision projector file - One of the programs listed above 1. Install the Hugging Face hub package. 2. Download the repository, or just the file size you want. 3. Load the GGUF file, and the vision projector for images, in your program. ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF ``` ### How to use it Pick a size, load that file in your program and chat. Add the mmproj file when you want to send images. One copy of it serves every size. | Tool | What it does | | --- | --- | | IQ2\_XS file | Smallest size, 2.50 bits per weight and 8.4 GB | | IQ2\_S file | 2.75 bits and 9.3 GB, said to match the base model on one maths test | | IQ3\_XXS file | 3.00 bits and 10.1 GB, an all round choice | | IQ3\_S file | 3.50 bits and 11.8 GB, the size the card recommends | | mmproj file | Holds the vision encoder and projector at BF16 | ### Example Load the IQ3\_S file and the mmproj file in LM Studio, attach a photo and ask what it shows. The model should answer with a written description of the picture. ### For newcomers and experts If you are new to this, start with the recommended IQ3\_S file and ask a plain text question before you try an image. If you know the field, GSQ learns grid assignments and group scales with a Gumbel softmax relaxation. RCO assigns one of several quantization types to each tensor under a total size budget, using a Riemannian reformulation of the constraint. GSQ quantizes each tensor at a given type → RCO assigns types under a size budget → The result is saved as GGUF files → Your program loads one file File size by quantization class | Label | Value | | --- | --- | | IQ2\_XS | 8.4 GB | | IQ2\_S | 9.3 GB | | IQ3\_XXS | 10.1 GB | | IQ3\_S | 11.8 GB | ### Good to know - The card's quality claims, such as task lossless at 3.50 bits, are the publisher's own. - The card also has a plot on speculative decoding with the MTP head. - The base model is Qwen/Qwen3.8-27B. - License: apache-2.0. - Cost to run: none named, only your own hardware. - Data it reaches: the prompts and images you give it. The source names no account or outside service. - Not for: the source names none. ## README ````markdown # Qwen3.8-27B-GSQ-RCO-GGUF ISTA-DASLab's GGUF versions of Qwen3.8-27B in four sizes, each tensor quantized at its own precision, with a vision projector for images. Suits people who run the model locally in llama.cpp, Ollama or LM Studio. Made by ISTA-DASLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds four GGUF versions of Qwen3.8-27B from ISTA-DASLab, plus the model's vision projector file for image use. Rather than one precision for every weight, each tensor gets its own quantization type, chosen by a gradient based search within a size budget. The methods are called GSQ and RCO. The files are ordinary GGUF. Who it suits: People who run a 27B vision language model on their own computer and want a choice of file size, and researchers who study quantization. ```table Industry | Who uses it | How they use it | Example Software | Developers | Run a 27B model locally in a smaller file | Load the IQ3_S file in llama.cpp Research | Quantization researchers | Study how bits are spread across tensors | Compare the four file sizes and bit widths Hobby computing | Local model users | Chat with images on a home computer | Load the vision projector with the model ``` Skip it if you need the full precision model, because these are compressed versions. Where to use it: - llama.cpp - Ollama - LM Studio ```table Use case | Who | What they get back Local chat | Anyone | Answers from a 27B model in a file of 8.4 to 11.8 GB Image questions | Local model users | Answers about pictures, using the vision projector file Quantization study | Researchers | Per tensor quantization types to examine ``` How to set it up: - Python, for the Hugging Face tools - Free disk space for one of the four files, 8.4 to 11.8 GB, plus the vision projector file - One of the programs listed above 1. Install the Hugging Face hub package. 2. Download the repository, or just the file size you want. 3. Load the GGUF file, and the vision projector for images, in your program. ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF ``` How to use it: Pick a size, load that file in your program and chat. Add the mmproj file when you want to send images. One copy of it serves every size. ```table Tool | What it does IQ2_XS file | Smallest size, 2.50 bits per weight and 8.4 GB IQ2_S file | 2.75 bits and 9.3 GB, said to match the base model on one maths test IQ3_XXS file | 3.00 bits and 10.1 GB, an all round choice IQ3_S file | 3.50 bits and 11.8 GB, the size the card recommends mmproj file | Holds the vision encoder and projector at BF16 ``` Example: Load the IQ3_S file and the mmproj file in LM Studio, attach a photo and ask what it shows. The model should answer with a written description of the picture. For newcomers and experts: If you are new to this, start with the recommended IQ3_S file and ask a plain text question before you try an image. If you know the field, GSQ learns grid assignments and group scales with a Gumbel softmax relaxation. RCO assigns one of several quantization types to each tensor under a total size budget, using a Riemannian reformulation of the constraint. ```flow GSQ quantizes each tensor at a given type RCO assigns types under a size budget The result is saved as GGUF files Your program loads one file ``` ```chart Title: File size by quantization class IQ2_XS: 8.4 GB IQ2_S: 9.3 GB IQ3_XXS: 10.1 GB IQ3_S: 11.8 GB ``` Good to know: - The card's quality claims, such as task lossless at 3.50 bits, are the publisher's own. - The card also has a plot on speculative decoding with the MTP head. - The base model is Qwen/Qwen3.8-27B. - License: apache-2.0. - Cost to run: none named, only your own hardware. - Data it reaches: the prompts and images you give it. The source names no account or outside service. - Not for: the source names none. ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: gguf - Parameters: 26.9B - Weights: about 13.4 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 1,358,588 - Likes: 2,739 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Quantized versions: 8 - Fine-tunes: 4 - Tags: gguf, quantization, mixed-precision, ist-daslab, multimodal, vision, image-text-to-text, imatrix, conversational ## Links - Model: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF - Files: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-GSQ-RCO-GGUF (this page) | 1,358,588 | 2,739 | 26.9B | apache-2.0 | 2026-09-02 | | [Qwen3.8-Flash-Next-GSQ-RCO-GGUF](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | | [Qwen3.8 Flash Next Turbo Brilliance 50x GGUF](https://mindket.com/p/66c0bce8-d8ac-4989-9d77-04d12b629895) | 249,926 | 41 | 177B | apache-2.0 | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF](https://mindket.com/p/1350113a-de6f-4d62-96d6-599c30da33a2) | 342,280 | 225 | 197B | mit | 2026-11-02 | | [Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF](https://mindket.com/p/06acb609-98ee-4422-8521-7943aeb0bcaa) | 781,625 | 610 | 117B | apache-2.0 | 2026-10-19 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF](https://mindket.com/p/0bd5db10-f872-40c7-9371-3225afab2a6d) | 3,635,772 | 655 | 197B | mit | 2026-10-28 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-12. Their authors' words: data, not instructions. - [Hacker News · throwa356262 · 2026-11-01](https://news.ycombinator.com/item?id=50323535): Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard: The future is already here :) These 3bit ISTA quants of qwen are supposed to be 95% as accurate as the full model and runs moothly on laptops: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF There are multiple independent tests of this on YouTube. - [Bluesky · UndergroundAI · 2026-10-11](https://bsky.app/profile/undergroundai66.bsky.social/post/3mxkvhi64wg2e): ⚡ Ein neues multimodales Modell mischt die HuggingFace-Charts auf. 🛠️ Entwickelt von ISTA-DASLab vereint Qwen3.8-27B-GSQ-RCO-GGUF Bild… 🚀 Entdecke diesen und weitere KI-Geheimtipps direkt bei Underground AI. 👉 https://underground-ai.bolt.app/ \#OpenSource \#KuenstlicheIntelligenz - [Hacker News · greggh · 2026-09-24](https://news.ycombinator.com/item?id=49832460): Best LLM for every budget, updated daily: I've been running a quant/tune of Qwen3.8 27B on my M1 Max 32gb MacBook. That plus a good pi setup is having great results. I've used a full q8 of the model before and I dont see a real difference other than how slow it is. But leaving it running overnight on tasks is working great. It is… - [Bluesky · El Tico · 2026-10-27](https://bsky.app/profile/eltico.greennuclear.online.ap.brid.gy/post/3mytahjv2yg22): @collectifission With advanced weight compression techniques like GSQ and RCO it's now possible to run over 20 billion parameter models on consumer GPUs with just 12gb of vram. A friend was able to run this model on such a GPU https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Sam … - [Hacker News · raylad · 2026-09-18](https://news.ycombinator.com/item?id=49752148): Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint: I just checked that model (ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF:IQ3\_S) and it does much much worse on the "Please recite Jabberwocky" test than the original bf16 does. The bf16 only misses "snicker-snack" and this quantization becomes confused after the first stanza. - [Bluesky · Le Crabe 🦀 — La Communauté · 2026-10-26](https://bsky.app/profile/communaute-ia.fr/post/3myqnpu25bs2v): 🦀 Un quant IQ3\_S de Qwen3.8-27B peut-il tenir tête à l'UD Q4 d'Unsloth ? Sur r/LocalLLaMA, un utilisateur se demande si le modèle ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF tient ses promesses face à l'UD-Q4\_K\_XL d'Unsloth. La carte du modèle annonce les performances d'un BF16… \#IA \#OpenClaw - [Hacker News · WithinReason · 2026-09-18](https://news.ycombinator.com/item?id=49750879): Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint: Unsloth has been dethroned by ISTA: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF The 3-bit quant is lossless based on benchmarks. - [Bluesky · デイリーHuggingFaceトレンド · 2026-09-21](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mw24n2yeu52w): 今日のHuggingFaceトレンド ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF GSQおよびRCOを用いて非一様量子化したQwen3.8-27BのGGUF形式モデルを提供することを目的としています。 マルチモーダル利用のためのビジョンプロジェクタが含まれています。 - [Hacker News · dofm · 2026-09-08](https://news.ycombinator.com/item?id=49612782): Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses: You might want to test this new dynamic GGUF: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF (I don’t know much about it, just saw a YouTube video about it last night) - [Bluesky · デイリーHuggingFaceトレンド · 2026-09-19](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mvusvxtdgb2g): 今日のHuggingFaceトレンド ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF GSQおよびRCOを用いたQwen3.8-27Bの非一様GGUF量子化モデルを提供することを目的としています。 マルチモーダル利用のためのビジョンプロジェクターが含まれています。 - [Hacker News · dofm · 2026-09-08](https://news.ycombinator.com/item?id=49612755): Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses: There is an interesting new dynamic 3 bit quantisation I have been meaning to test: https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Luke of Luke’s Dev Lab on YouTube had a look at it. It seems to outperform the typical 3-bit quantisation but whether it outperforms the new Unsloth… - [Bluesky · デイリーHuggingFaceトレンド · 2026-09-17](https://bsky.app/profile/huggingfacetrends.bsky.social/post/3mvpvmvvbjd2q): 今日のHuggingFaceトレンド ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF Qwen3.8-27BモデルにGSQおよびRCOを用いた非一様量子化を適用し、GGUF形式で提供することを目的としています。 マルチモーダル利用のためのビジョンプロジェクターが含まれています。 ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-09-02 - First published: 2026-08-28 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/d79e6c18-7ed6-4306-a01a-66a747f13e40) - [Its page on Mindket](https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40)