--- title: "Qwen3.6-35B-A3B 2-bit GSQ GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/6d3048f5-cd06-4079-a604-ac9311e5fb97" updated: "2026-11-13" --- # Qwen3.6-35B-A3B 2-bit GSQ GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs a 35B Qwen model in a 2-bit GSQ GGUF conversion with a separate vision projector for llama.cpp. For people who want a large model in a small file on CPU or a modest GPU. Qwen3.6-35B-A3B 2-bit GSQ GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by chtisgit ## About ### What it is This is a GGUF conversion of the Qwen3.6-35B-A3B 2-bit GSQ checkpoint from ISTA-DASLab, repackaged into stock llama.cpp types. Nothing here is newly quantized or trained, since the weights come from the DASLab checkpoint. The files are a text-model GGUF plus a separate vision-projector GGUF, so the model can read images as well as text. ### Who it suits It suits people who want to run a 35B model with routed experts in a small file, using standard llama.cpp on a CPU or a modest graphics card. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI | Hobbyists with a mid-range GPU | Run a large model from a small 2-bit file | The card tested it with part of the layers offloaded to an RTX 3060 | | Software development | Teams building on llama.cpp | Try 2-bit GSQ weights without a custom runtime | The files load in a current stock llama.cpp build | | Research | Quantization researchers | Study how GSQ 2-bit maps onto Q2\_0 | The card documents the mapping and its small errors | You can skip it if you cannot run a current llama.cpp build, because older releases cannot load it correctly. ### Where to use it - llama.cpp on Windows, macOS or Linux - CPU-only runs, which the card tested and which avoid the CUDA crash - NVIDIA GPUs with llama.cpp build b11390 or newer | Use case | Who | What they get back | | --- | --- | --- | | Chat with a large local model | Local model users | A 35B model that fits in a small text file | | Ask about an image | Users with the projector loaded | Text read or described from the picture | | Check a conversion | Researchers | A documented mapping from GSQ types to llama.cpp types | ### How to set it up You need a current llama.cpp build and both model files. - A current llama.cpp build, because Q2\_0 is a recent type that older releases cannot load - On NVIDIA GPUs, build b11390 or newer - Python with the Hugging Face tools, to download the files - Room for the text file of 11.38 GiB plus the small projector 1. Install the Hugging Face tools with Python. 2. Download the repository files with the second command below. 3. Update llama.cpp to a current build, and use b11390 or newer on NVIDIA GPUs. 4. Load the text model together with the projector file. ```sh pip install -U huggingface_hub hf download chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF ``` ### How to use it The card shows no run command, so follow the llama.cpp documentation for loading a text model with a vision projector. Load the text GGUF for chat, and add the projector GGUF when you want the model to read images. | Tool | What it does | | --- | --- | | Qwen3.6-35B-A3B-GSQ-hybrid.gguf | The text model | | mmproj-BF16.gguf | The vision projector for image input | ### Example You load both files in a current llama.cpp build and show the model a newspaper test image. The card reports that runs on CPU and with partial offload to an RTX 3060 both read the headline MEN WALK ON MOON correctly. ### For newcomers and experts Think of it as a large model squeezed into a small file, with a second file that lets it see pictures. A first small try is to download both files and load them in a current llama.cpp build. The routed experts are stored as Q2\_0. The attention layers, the shared expert and the LM head are stored as Q8\_0. A GSQ 2-bit code maps to Q2\_0 by reversing the code order and negating the scale. Groups of 128 weights become two Q2\_0 blocks of 64, with no dequantize or requantize step. The only numerical change is the BF16 scale becoming FP16. One MTP layer is left out. ```text Q2_0 code = 3 - c Q2_0 scale = FP16(-s) ``` Take the GSQ 2-bit checkpoint → Map routed experts to Q2\_0 by reversing codes → Copy INT8 weights into Q8\_0 blocks → Write a text GGUF and a projector GGUF → Load both in a current llama.cpp build ### Good to know The two files are described in the card as follows. The two model files | Property | Value | | --- | --- | | Text model | Qwen3.6-35B-A3B-GSQ-hybrid.gguf | | Text-model size | 11.38 GiB | | Vision projector | mmproj-BF16.gguf | | Projector size | 860.9986 MiB | - Crash on older builds: older CUDA builds fail on prompts of certain lengths with an illegal memory access error, and CPU-only builds are not affected - Tests: the card ran load and short-generation checks, not benchmarks or perplexity runs - Left out: one MTP layer for speculative decoding - Size: 34.7B parameters - License: Apache 2.0 - Cost to run: none named - Data it reaches: none named beyond the files you download - Not for: older llama.cpp releases, which cannot load or run it correctly ## README ````markdown # Qwen3.6-35B-A3B 2-bit GSQ GGUF Runs a 35B Qwen model in a 2-bit GSQ GGUF conversion with a separate vision projector for llama.cpp. For people who want a large model in a small file on CPU or a modest GPU. Made by chtisgit on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This is a GGUF conversion of the Qwen3.6-35B-A3B 2-bit GSQ checkpoint from ISTA-DASLab, repackaged into stock llama.cpp types. Nothing here is newly quantized or trained, since the weights come from the DASLab checkpoint. The files are a text-model GGUF plus a separate vision-projector GGUF, so the model can read images as well as text. Who it suits: It suits people who want to run a 35B model with routed experts in a small file, using standard llama.cpp on a CPU or a modest graphics card. ```table Industry | Who uses it | How they use it | Example Local AI | Hobbyists with a mid-range GPU | Run a large model from a small 2-bit file | The card tested it with part of the layers offloaded to an RTX 3060 Software development | Teams building on llama.cpp | Try 2-bit GSQ weights without a custom runtime | The files load in a current stock llama.cpp build Research | Quantization researchers | Study how GSQ 2-bit maps onto Q2_0 | The card documents the mapping and its small errors ``` You can skip it if you cannot run a current llama.cpp build, because older releases cannot load it correctly. Where to use it: - llama.cpp on Windows, macOS or Linux - CPU-only runs, which the card tested and which avoid the CUDA crash - NVIDIA GPUs with llama.cpp build b11390 or newer ```table Use case | Who | What they get back Chat with a large local model | Local model users | A 35B model that fits in a small text file Ask about an image | Users with the projector loaded | Text read or described from the picture Check a conversion | Researchers | A documented mapping from GSQ types to llama.cpp types ``` How to set it up: You need a current llama.cpp build and both model files. - A current llama.cpp build, because Q2_0 is a recent type that older releases cannot load - On NVIDIA GPUs, build b11390 or newer - Python with the Hugging Face tools, to download the files - Room for the text file of 11.38 GiB plus the small projector 1. Install the Hugging Face tools with Python. 2. Download the repository files with the second command below. 3. Update llama.cpp to a current build, and use b11390 or newer on NVIDIA GPUs. 4. Load the text model together with the projector file. ```sh pip install -U huggingface_hub hf download chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF ``` How to use it: The card shows no run command, so follow the llama.cpp documentation for loading a text model with a vision projector. Load the text GGUF for chat, and add the projector GGUF when you want the model to read images. ```table Tool | What it does Qwen3.6-35B-A3B-GSQ-hybrid.gguf | The text model mmproj-BF16.gguf | The vision projector for image input ``` Example: You load both files in a current llama.cpp build and show the model a newspaper test image. The card reports that runs on CPU and with partial offload to an RTX 3060 both read the headline MEN WALK ON MOON correctly. For newcomers and experts: Think of it as a large model squeezed into a small file, with a second file that lets it see pictures. A first small try is to download both files and load them in a current llama.cpp build. The routed experts are stored as Q2_0. The attention layers, the shared expert and the LM head are stored as Q8_0. A GSQ 2-bit code maps to Q2_0 by reversing the code order and negating the scale. Groups of 128 weights become two Q2_0 blocks of 64, with no dequantize or requantize step. The only numerical change is the BF16 scale becoming FP16. One MTP layer is left out. ```text Q2_0 code = 3 - c Q2_0 scale = FP16(-s) ``` ```flow Take the GSQ 2-bit checkpoint Map routed experts to Q2_0 by reversing codes Copy INT8 weights into Q8_0 blocks Write a text GGUF and a projector GGUF Load both in a current llama.cpp build ``` Good to know: The two files are described in the card as follows. ```table Title: The two model files Property | Value Text model | Qwen3.6-35B-A3B-GSQ-hybrid.gguf Text-model size | 11.38 GiB Vision projector | mmproj-BF16.gguf Projector size | 860.9986 MiB ``` - Crash on older builds: older CUDA builds fail on prompts of certain lengths with an illegal memory access error, and CPU-only builds are not affected - Tests: the card ran load and short-generation checks, not benchmarks or perplexity runs - Left out: one MTP layer for speculative decoding - Size: 34.7B parameters - License: Apache 2.0 - Cost to run: none named - Data it reaches: none named beyond the files you download - Not for: older llama.cpp releases, which cannot load or run it correctly ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: llama.cpp - Parameters: 34.7B - Weights: about 17.3 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 9,309 - Likes: 18 - License: apache-2.0 - Based on: ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ - Tags: llama.cpp, gguf, qwen, quantization, image-text-to-text, conversational ## Links - Model: https://huggingface.co/chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF - Files: https://huggingface.co/chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF on Hugging Face](https://huggingface.co/chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.6-35B-A3B 2-bit GSQ GGUF (this page) | 9,309 | 18 | 34.7B | apache-2.0 | 2026-10-22 | | [Qwen3.8-Flash-Coder APEX-I-MiniPlus V2.1 GGUF](https://mindket.com/p/238d4a60-b5ce-4e02-a828-5e9dce37a102) | 1,585 | 8 | 42.6B | apache-2.0 | 2026-10-11 | | [Qwopus3.8-27B-Flash-GGUF](https://mindket.com/p/509fda01-c99c-450a-8d3e-2444ec0c6af7) | 2,153 | 248 | 461M | apache-2.0 | 2026-11-07 | | [Qwen3.6-35B-A3B-2Bit-GSQ](https://mindket.com/p/1419ceb9-6ab2-48e2-aec8-e0401259317e) | 974 | 37 | 35.5B | apache-2.0 | 2026-08-14 | | [Qwen3.8-27B-SLQ-GGUF](https://mindket.com/p/c549f05b-1d25-49d4-868a-58981cf20769) | 13,478 | 60 | 27.3B | apache-2.0 | 2026-10-27 | | [MiMo-V2.6-Distill-Qwen-9B-GSQ-RCO-GGUF](https://mindket.com/p/81b8b624-a1fa-4440-b7f9-b44fdafef92b) | 1,548 | 2 | 9B | apache-2.0 | 2026-10-15 | Numbers as Mindket last read them, on 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-10-22 - First published: 2026-09-21 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF on Hugging Face](https://huggingface.co/chtisgit/Qwen3.6-35B-A3B-2Bit-GSQ-hybrid-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/6d3048f5-cd06-4079-a604-ac9311e5fb97) - [Its page on Mindket](https://mindket.com/p/6d3048f5-cd06-4079-a604-ac9311e5fb97)