Qwen3.8-27B GGUF (ShapeLearn Quantized)

Qwen3.8-27B GGUF (ShapeLearn Quantized)

Runs Qwen3.8-27B as GGUF files quantized by ByteShape's ShapeLearn, in five GPU-sized options with vision input and MTP speculative decoding in llama.cpp. For people running it on their own GPU.

Made by byteshape on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds GGUF copies of Qwen3.8-27B quantized with ByteShape's ShapeLearn. ShapeLearn learns the best datatype for each tensor so quality holds up even at low bit lengths. The release replaces the earlier ShapeLearn-Lite versions, and each of its five models was scored against the BF16 baseline.

Who it suits: It suits people who run Qwen3.8-27B on their own GPU through llama.cpp and want to pick a size and quality trade-off that fits their card.

Industry | Who uses it | How they use it | Example
Software | Developers with a consumer GPU | Pick the model size that fits their GPU memory | Choose a smaller file for a card with less memory
Research | People comparing quantizations | Read the quality-versus-speed chart on the card | Compare ShapeLearn files with other public GGUF copies
Vision work | Users who send images | Run the model with the vision projector | Ask a question about a picture

People who want the unquantized weights can skip it, since every file here is quantized.

Where to use it:

• llama.cpp, through llama-server
• The web interface that llama-server serves on your own machine
• GPUs such as the RTX 5090, which the card benchmarks, along with the 4090, 4080, 5060 Ti, 3090 and RTX PRO 6000 Blackwell
• Windows, macOS or Linux, where you can download the files with Python

Use case | Who | What they get back
Local chat and coding | Developers | A running llama-server with the model chosen by its Model Tag
Vision questions | Users with images | Answers that use the auto-downloaded vision projector
Faster decoding | Speed-minded users | MTP speculative decoding with no separate draft model

How to set it up: You need these first:

• llama.cpp, with build b10658 or newer if you want DFlash 2
• A GPU with room for the model you choose, from about 8.8 GB for the smallest of the five
• Python and the huggingface_hub package, if you want to download the files yourself

1. Pick a model from the card's table and copy its Model Tag.
2. Make sure your llama.cpp build is new enough.
3. Run llama-server with the tag, as shown below.

llama-server -hf <MODEL_TAG> --mmproj-auto --spec-type draft-mtp --spec-draft-n-max 3

llama-server --help | grep spec-type

pip install -U huggingface_hub
hf download byteshape/Qwen3.8-27B-GGUF

How to use it: Run llama-server with a Model Tag and open the web interface it serves on your machine. On the first run, llama.cpp fetches the model and the vision projector. The draft-mtp option turns on MTP speculative decoding, and the draft count option sets how many tokens are drafted per step.

Tool | What it does
--mmproj-auto | Loads the vision projector so the model can read images
--spec-type draft-mtp | Turns on MTP speculative decoding using the head inside the GGUF
--spec-draft-n-max | Sets how many tokens are drafted per step

Example: You copy the Model Tag of one of the five models and run llama-server with it. llama.cpp downloads the model and the vision projector, then you open the web interface and chat. The card says speculative decoding adds another 1.4 to 1.8 times on top of plain decoding.

For newcomers and experts: Quantizing shrinks a model so it fits a smaller GPU, at some cost in quality. A first small step is to pick the smallest model in the table and run the command above.

ShapeLearn picks a datatype per tensor, so each file mixes quantization types instead of using one for all. The card plots quality against tokens per second on an RTX 5090, with quality as the average score relative to BF16 on its benchmark suite. The speed figure is single-stream decoding with plain next-token prediction. The MTP head sits inside the GGUF, so you need no separate draft model.

Pick a model from the table
Copy its Model Tag
Run llama-server with the tag
llama.cpp fetches model and projector
Chat in the web interface

Good to know:

• Size: 27.3B parameters per Hugging Face, about 13.7 GB at 4-bit as an estimate, and the card's own files start at 8.8 GB
• Evaluation: scored against BF16 on tests of math, code, knowledge, following instructions and tool use, in thinking and instruct modes
• Versions: Hugging Face lists 1 quantized version made by other people
• DFlash 2: it needs a llama.cpp build of b10658 or newer, and draft-mtp works on any build from that release on

Title: Model size in GB, first three of five models
GPU-1: 8.8 GB
GPU-2: 9.9 GB
GPU-3: 10.4 GB

• License: apache-2.0
• Cost to run: none named, no API key or account
• Data it reaches: on the first run llama.cpp fetches the model and its vision projector, and the web interface stays on your own machine
• Not for: DFlash 2 on llama.cpp builds older than b10658, since that release added the support

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 27.3B
• Weights: about 13.7 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 2,083,628
• Likes: 216
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 1
• Tags: transformers, gguf, qwen3.8, byteshape, shapelearn, dflash, agentic, image-text-to-text, conversational

Links

• Model: https://huggingface.co/byteshape/Qwen3.8-27B-GGUF
• Files: https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download byteshape/Qwen3.8-27B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.