Qwen3.5-9B GGUF (ShapeLearn)

Qwen3.5-9B GGUF (ShapeLearn)

GGUF copies of Qwen3.5-9B, a model that reads text and images, compressed with ByteShape's ShapeLearn for llama.cpp. Variants for GPUs and CPUs, from about 3 GB to 5.7 GB.

Made by byteshape on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3.5-9B GGUF is a set of quantized copies of Qwen3.5-9B, a 9B model that accepts text and images. ByteShape made them with ShapeLearn, which learns the best datatype for each tensor so quality holds up at low bit counts. Variants exist for both GPUs and CPUs.

Who it suits: It suits people who run a vision-capable model locally in llama.cpp and want a size that fits their GPU or CPU. The card gives a simple rule for choosing a file.

Title: Who uses it
Industry | Who uses it | How they use it | Example
Software | Application developers | Serve a vision model locally through llama.cpp | Start llama-server with a model tag
AI tooling | Local-model users | Pick a size for a GPU, a CPU or a small board | Choose a 3 GB file for a small graphics card
Research | Engineers | Compare quality against speed across sizes | Read the card's chart against an Unsloth baseline

People who use Ollama can skip it for now, since the card says Ollama does not support this model's GGUF files.

Where to use it:

• llama.cpp, through llama-server
• LM Studio, which the card suggests
• A web interface on localhost once the server runs
• Windows, macOS or Linux

Title: Top jobs
Use case | Who | What they get back
Local chat | Individuals | Replies from a 9B model on their own machine
Questions about images | Developers | Answers that use a picture, through the vision projector
Size selection | Local-model users | A file whose size and speed fit their hardware

How to set it up:

• Python on your computer, for the download
• The huggingface_hub package
• llama.cpp installed
• Memory for the file you choose, from about 3 GB up

1. Install the Hugging Face hub package.
2. Download the files, or skip this and let llama.cpp fetch one on first run.
3. Pick a size from the table below.
4. Start llama-server with that file's model tag.

pip install -U huggingface_hub
hf download byteshape/Qwen3.5-9B-GGUF

Title: GPU files by size
Model ID | Bits per weight | File size
GPU-1 | 2.81 | 3.15 GB
GPU-2 | 3.00 | 3.37 GB
GPU-3 | 3.15 | 3.53 GB
GPU-4 | 3.60 | 4.04 GB
GPU-5 | 4.20 | 4.71 GB
GPU-6 | 4.43 | 4.97 GB
GPU-7 | 5.10 | 5.72 GB

How to use it: Copy a model tag and hand it to llama-server. The vision projector downloads on the first run along with the model. The server then offers a web interface on localhost.

llama-server -hf <MODEL_TAG> --mmproj-auto

A tag has this shape, using the first GPU file as the example.

byteshape/Qwen3.5-9B-GGUF:Qwen3.5-9B-IQ3_S-2.81bpw.gguf

Example: You pick GPU-4 for a mid-range card, start llama-server with its tag and wait for the download. In the web interface you attach a photo and ask what it shows. The model answers using both the picture and your question.

For newcomers and experts: If you are new, take the largest file your memory allows, since bigger files keep more quality. A first try is one image and one short question in the web interface.

For experts, GPU files mix KQ and IQ quantization for speed, and CPU files lean mostly on KQ. Quality is scored on seven benchmarks that cover function calling, coding, math and general knowledge, in thinking and instruct modes. The card plots quality against tokens per second, with Unsloth as the baseline.

Pick a file for your hardware
Copy its model tag
Run llama-server with the tag
Chat or send images in the web interface

Good to know:

• Rule of thumb: the card says to take the best quality at the speed you need, or the quickest file that still reaches your quality bar.
• Ollama: the card says it does not support this model's GGUF files.
• Labels: names such as IQ4_XS only describe the main method and bit length.
• License: Apache 2.0.
• Cost to run: none named.
• Data it reaches: it downloads files from Hugging Face, and the server runs on your own machine.
• Not for: Ollama users.

Title: GPU file size by model ID
GPU-1: 3.15 GB
GPU-3: 3.53 GB
GPU-5: 4.71 GB
GPU-7: 5.72 GB

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 9B
• Weights: about 4.5 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 3,023
• Likes: 41
• License: apache-2.0
• Based on: Qwen/Qwen3.5-9B
• Tags: transformers, gguf, qwen3.5, byteshape, image-text-to-text, conversational

Links

• Model: https://huggingface.co/byteshape/Qwen3.5-9B-GGUF
• Files: https://huggingface.co/byteshape/Qwen3.5-9B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download byteshape/Qwen3.5-9B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.