Qwen3.6-35B-A3B GGUF (ShapeLearn Quantized)

Qwen3.6-35B-A3B GGUF (ShapeLearn Quantized)

Quantized GGUF versions of Qwen3.6-35B-A3B that read images and text, with CPU and GPU variants from about 9 GB, made with ByteShape's ShapeLearn. For developers running llama.cpp.

Made by byteshape on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF versions of Qwen3.6-35B-A3B made with ByteShape's ShapeLearn. The method learns the best data type for each tensor, so quality holds up even at low bit counts. The model reads text and images, and the files come in separate CPU and GPU variants.

Who it suits: Developers who want a large vision-capable model on their own hardware and are ready to choose a size for their GPU or CPU. The card has benchmark plots for several desktop GPUs, CPUs and a 16 GB Raspberry Pi 5.

Title: Who uses it, and how
Industry | Who uses it | How they use it | Example
Local AI | Hobbyists with a gaming GPU | Pick a GPU variant sized for their card | Run the 13 GB GPU-2 file on a mid-range card
Software | Developers | Serve a vision model through llama.cpp | Start llama-server and send an image with a question
Edge computing | Embedded tinkerers | Try a CPU variant on small machines | Test a KQ file on a CPU-only computer
Research | ML engineers | Compare quality against speed across sizes | Use the card's selection rule on two sizes

Skip it if you use Ollama, because the card says Ollama does not support this model's GGUF files as of its release.

Where to use it:

• llama.cpp, using llama-server with its built-in web interface
• LM Studio, which the card suggests as an alternative
• Windows, macOS or Linux for downloading the files, with Python

Use case | Who | What they get back
Image questions | Developers | Text answers about a picture you attach
Text chat | Hobbyists | Replies in a local web interface
Size tuning | Engineers | A model whose size and speed fit your hardware

How to set it up: You need Python and pip to download the repository, and llama.cpp to run it. llama.cpp fetches the model and vision projector on its first run.

• Python and pip
• llama.cpp
• Disk space for the variant you pick, from about 9 GB

1. Choose a variant from the card's GPU or CPU table.
2. Copy its model tag.
3. Pass the tag to llama-server.

pip install -U huggingface_hub
hf download byteshape/Qwen3.6-35B-A3B-GGUF

Title: GPU variants
Model ID | Bits per weight | Size
GPU-1 | 2.17 | 9.42 GB
GPU-2 | 3.00 | 13 GB
GPU-3 | 3.48 | 15.1 GB
GPU-4 | 3.93 | 17 GB
GPU-5 | 4.15 | 18 GB

How to use it: Run llama-server with the model tag you copied, plus the option for the vision projector. Then open the web interface on your own machine. Choose the model with the best quality at the speed you need, or the fastest that still meets your quality bar.

llama-server -hf <MODEL_TAG> --mmproj-auto

Tool | What it does
llama-server -hf | Downloads the model on first run and starts a local server
--mmproj-auto | Fetches the vision projector so images work
Model tag | The name you copy from the card's table
LM Studio | An alternative runner the card suggests when Ollama will not work

Example: You start llama-server with the tag of the GPU-3 file, open its web page, attach a photo and ask what is in it. The model answers in text, using the vision projector it downloaded.

For newcomers and experts: If you are new, pick the mid-sized GPU-2 or GPU-3 file if you have a gaming graphics card, or a CPU file if you do not. Run the server command and open the web page it starts.

For people in the field, the card labels CPU models as KQ, mostly Q-K quantization, and GPU models as IQ, a hybrid of KQ and IQ for better throughput. Quality is scored on six benchmarks, including function calling, HumanEval and GSM8K, in thinking and instruct modes.

Choose a CPU or GPU variant
Copy its model tag
Run llama-server with the tag
Open the web interface
Chat or attach an image

Good to know:

Title: GPU variant size in GB
GPU-1: 9.42 GB
GPU-2: 13 GB
GPU-3: 15.1 GB
GPU-4: 17 GB
GPU-5: 18 GB

• License: Apache 2.0
• Cost to run: none named, you use your own computer
• Data it reaches: llama.cpp pulls the weights and the vision file from Hugging Face on the first launch, then reads any images you send
• Not for: Ollama, which the card says does not support these files yet
• The card's CPU table is not shown on this page

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 34.7B
• Weights: about 17.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 5,517
• Likes: 49
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Tags: transformers, gguf, qwen3.6, byteshape, image-text-to-text, conversational

Links

• Model: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-GGUF
• Files: https://huggingface.co/byteshape/Qwen3.6-35B-A3B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download byteshape/Qwen3.6-35B-A3B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.