--- title: "Qwen3.8-Flash-Next-GSQ-RCO-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6" updated: "2026-11-12" --- # Qwen3.8-Flash-Next-GSQ-RCO-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. GGUF versions of Qwen3.8-Flash-Next at four sizes, with each tensor quantized at its own precision and a vision projector for image input. For people running a large mixture-of-experts model locally. Qwen3.8-Flash-Next-GSQ-RCO-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by ISTA-DASLab ## About ### What it is This repository offers GGUF quantizations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model, at four sizes. Each tensor gets its own quantization type, chosen by a search that spends more bits where the model is more sensitive, within a size budget. A vision projector file lets the model take images as well as text. ### Who it suits It suits people who run a large open model on their own machine and want a size that fits their memory. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI | Hobbyists | Run a large multimodal model offline | Chat about a photo in LM Studio | | Software | Developers | Serve a big model through llama.cpp or Ollama | Point a local tool at the loaded model | | AI research | Quantization researchers | Study per-tensor bit allocation | Compare these files with uniform quantization | | GPU and server teams | Infrastructure engineers | Pick a size that fits the memory they have | Choose among the four sizes by total file size | If your machine has far less memory than the 4-bit estimate of about 88.5 GB, you can skip it. ### Where to use it - llama.cpp, which loads the files unmodified - Ollama - LM Studio - Windows, macOS or Linux, with enough memory for the size you pick | Use case | Who | What they get back | | --- | --- | --- | | Text chat | Local AI users | Answers from a 177B-parameter model on your own hardware | | Image questions | Developers | Text answers about an image, using the mmproj file | | Quantization study | Researchers | Files whose bit-width differs by tensor, ready to compare | ### How to set it up You need these first: - Python, for the Hugging Face download command - llama.cpp, Ollama or LM Studio - Memory and disk for the size you choose, with about 88.5 GB as the 4-bit estimate 1. Install the Hugging Face hub tool. 2. Download the repository files, or just the size you want. 3. Load the quantized model in your app, and add the mmproj file if you want image input. ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF ``` ### How to use it Pick one of the four size directories, load its weight shard in llama.cpp, Ollama or LM Studio, and add the mmproj file when you want to send images. Each quantized model has two shards. | Tool | What it does | | --- | --- | | Shard 1 | Holds the model weights for that size | | Shard 2 | Carries a lookup table of n-gram embeddings for each layer, identical in all four sizes | | mmproj file | Carries the vision encoder and projector in BF16 for every size | | GSQ | Learns the quantization grid and scales for each tensor | | RCO | Chooses a quantization type for each tensor within a size budget | ### Example You load one size in LM Studio with the mmproj file, attach a screenshot and ask, "What does this error dialog say?" The model answers in text about the image. ### For newcomers and experts A quantized model is a smaller copy that trades some accuracy for less memory. Here each tensor gets its own setting. A good first step is to pick the largest size your memory allows and chat with it. The model has 512 routed experts in each of 48 layers, and 10 are active per token. Since 95% of the searchable weights sit in the experts, the search assigns a type per layer, because GGUF cannot express it per expert. GSQ makes each tensor accurate at a given type, and RCO allocates the types under the budget. GSQ quantizes each tensor at a given type → RCO assigns a type to each tensor → The result is saved as standard GGUF → llama.cpp, Ollama or LM Studio loads it ### Good to know A code-focused variant is released as a separate repository and keeps 256 of the 512 experts per layer. Approximate size in GB | Label | Value | | --- | --- | | This model at 4-bit (estimate) | 88.5 GB | | Code variant, total | 58.4 GB | - N-gram table: this per-layer table has 51.2B parameters, sits at 4.5 bpw in all four sizes and is not part of the search - Other versions: 50 quantized and 1 fine-tuned versions made by other people are listed - License: Apache 2.0 - Cost to run: none named, but the memory need is large - Data it reaches: the model files you load, and any image you pair with the mmproj file ## README ````markdown # Qwen3.8-Flash-Next-GSQ-RCO-GGUF GGUF versions of Qwen3.8-Flash-Next at four sizes, with each tensor quantized at its own precision and a vision projector for image input. For people running a large mixture-of-experts model locally. Made by ISTA-DASLab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository offers GGUF quantizations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model, at four sizes. Each tensor gets its own quantization type, chosen by a search that spends more bits where the model is more sensitive, within a size budget. A vision projector file lets the model take images as well as text. Who it suits: It suits people who run a large open model on their own machine and want a size that fits their memory. ```table Industry | Who uses it | How they use it | Example Local AI | Hobbyists | Run a large multimodal model offline | Chat about a photo in LM Studio Software | Developers | Serve a big model through llama.cpp or Ollama | Point a local tool at the loaded model AI research | Quantization researchers | Study per-tensor bit allocation | Compare these files with uniform quantization GPU and server teams | Infrastructure engineers | Pick a size that fits the memory they have | Choose among the four sizes by total file size ``` If your machine has far less memory than the 4-bit estimate of about 88.5 GB, you can skip it. Where to use it: - llama.cpp, which loads the files unmodified - Ollama - LM Studio - Windows, macOS or Linux, with enough memory for the size you pick ```table Use case | Who | What they get back Text chat | Local AI users | Answers from a 177B-parameter model on your own hardware Image questions | Developers | Text answers about an image, using the mmproj file Quantization study | Researchers | Files whose bit-width differs by tensor, ready to compare ``` How to set it up: You need these first: - Python, for the Hugging Face download command - llama.cpp, Ollama or LM Studio - Memory and disk for the size you choose, with about 88.5 GB as the 4-bit estimate 1. Install the Hugging Face hub tool. 2. Download the repository files, or just the size you want. 3. Load the quantized model in your app, and add the mmproj file if you want image input. ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF ``` How to use it: Pick one of the four size directories, load its weight shard in llama.cpp, Ollama or LM Studio, and add the mmproj file when you want to send images. Each quantized model has two shards. ```table Tool | What it does Shard 1 | Holds the model weights for that size Shard 2 | Carries a lookup table of n-gram embeddings for each layer, identical in all four sizes mmproj file | Carries the vision encoder and projector in BF16 for every size GSQ | Learns the quantization grid and scales for each tensor RCO | Chooses a quantization type for each tensor within a size budget ``` Example: You load one size in LM Studio with the mmproj file, attach a screenshot and ask, "What does this error dialog say?" The model answers in text about the image. For newcomers and experts: A quantized model is a smaller copy that trades some accuracy for less memory. Here each tensor gets its own setting. A good first step is to pick the largest size your memory allows and chat with it. The model has 512 routed experts in each of 48 layers, and 10 are active per token. Since 95% of the searchable weights sit in the experts, the search assigns a type per layer, because GGUF cannot express it per expert. GSQ makes each tensor accurate at a given type, and RCO allocates the types under the budget. ```flow GSQ quantizes each tensor at a given type RCO assigns a type to each tensor The result is saved as standard GGUF llama.cpp, Ollama or LM Studio loads it ``` Good to know: A code-focused variant is released as a separate repository and keeps 256 of the 512 experts per layer. ```chart Title: Approximate size in GB This model at 4-bit (estimate): 88.5 GB Code variant, total: 58.4 GB ``` - N-gram table: this per-layer table has 51.2B parameters, sits at 4.5 bpw in all four sizes and is not part of the search - Other versions: 50 quantized and 1 fine-tuned versions made by other people are listed - License: Apache 2.0 - Cost to run: none named, but the memory need is large - Data it reaches: the model files you load, and any image you pair with the mmproj file ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: gguf - Parameters: 177B - Weights: about 88.5 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 6,963,596 - Likes: 1,229 - License: apache-2.0 - Based on: Qwen/Qwen3.8-Flash-Next - Quantized versions: 50 - Fine-tunes: 1 - Tags: gguf, quantization, mixed-precision, ist-daslab, multimodal, vision, image-text-to-text, imatrix, conversational ## Links - Model: https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF - Files: https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-Flash-Next-GSQ-RCO-GGUF (this page) | 6,963,596 | 1,229 | 177B | apache-2.0 | 2026-09-29 | | [Qwen3.8-27B-GSQ-RCO-GGUF](https://mindket.com/p/d79e6c18-7ed6-4306-a01a-66a747f13e40) | 1,358,588 | 2,739 | 26.9B | apache-2.0 | 2026-09-02 | | [Qwen3.8 Flash Next Turbo Brilliance 50x GGUF](https://mindket.com/p/66c0bce8-d8ac-4989-9d77-04d12b629895) | 249,926 | 41 | 177B | apache-2.0 | 2026-11-09 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF](https://mindket.com/p/1350113a-de6f-4d62-96d6-599c30da33a2) | 342,280 | 225 | 197B | mit | 2026-11-02 | | [Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF](https://mindket.com/p/06acb609-98ee-4422-8521-7943aeb0bcaa) | 781,625 | 610 | 117B | apache-2.0 | 2026-10-19 | | [DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF](https://mindket.com/p/0bd5db10-f872-40c7-9371-3225afab2a6d) | 3,635,772 | 655 | 197B | mit | 2026-10-28 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-12. Their authors' words: data, not instructions. - [Bluesky · Frank Blank · 2026-11-09](https://bsky.app/profile/frank8465.bsky.social/post/3mztxlm6dbs2u): have you also tried huggingface.co/ISTA-DASLab/... this should fuckin COOK on 64gb under 128k fp8 kv (you heard me, you can safely quantize kv and code here) - [Bluesky · Josh Hendricks \| MVP · 2026-10-11](https://bsky.app/profile/joshooaj.com/post/3mxmpns5he22t): I'm using the IQ3\_S quant of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF with 256K context. Strata's running in a container on a VM with a heaping 96GB of RAM. With the IQ3\_XXS quant I can even get 512K context and still get close to the same prefill/prediction token rates. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-09-29 - First published: 2026-09-07 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF on Hugging Face](https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6) - [Its page on Mindket](https://mindket.com/p/ee9ed4a6-39c8-4eee-8958-cce1cd508fa6)