Qwen3.8-27B-AWQ-INT4

Qwen3.8-27B-AWQ-INT4

A 4-bit AWQ quantization of Qwen3.8-27B by cyankiwi, a 27B vision-language model with thinking control. Weights are 21.02 GB and load in Transformers, vLLM or SGLang.

Made by cyankiwi on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 4-bit AWQ quantization of Qwen3.8-27B made by cyankiwi. The base model is a dense 27B vision-language model that reads text, images and video and has switchable thinking. The quantized weights come to 21.02 GB and use the Hugging Face Transformers format. They were calibrated on cyankiwi's own calibration dataset.

Who it suits: It suits developers who deploy open models for coding, research or agent work and want a smaller copy of a 27B model that still reads images and video.

Industry | Who uses it | How they use it | Example
Software development | Developers who build coding assistants | Run a 27B model for coding and agent tasks from a smaller file | The base model targets coding and multi-step agent tasks
Research and analysis | Analysts | Ask questions about images and videos | A vision-language model reads the picture along with the question
Platform engineering | Teams that serve open models | Serve the weights through vLLM or SGLang | The files come in the Transformers format

You can skip it if you need the unquantized weights, since quantization changes the weights and outputs can differ from the original.

Where to use it:

• Hugging Face Transformers
• vLLM, SGLang and TokenSpeed, which the card names as compatible
• Windows, macOS or Linux with Python, to download the files
• cyankiwi private endpoints, through the deploy button on the card

Use case | Who | What they get back
Coding and agent work | Developers | A 27B model that fits a smaller memory budget
Image and video questions | Analysts | Answers that take the visual input into account
Self-hosted serving | Platform teams | An endpoint on a runtime they already use

How to set it up: You need Python, a supported runtime and room for 21.02 GB of weights.

• Python with the huggingface_hub package
• A runtime such as Transformers, vLLM, SGLang or TokenSpeed
• Enough memory for about 21.02 GB of weights, plus working space

1. Install the Hugging Face tools with Python.
2. Download the model files with the second command below.
3. Load the folder in your runtime of choice.
4. Send text, images or video, and read the answer.

pip install -U huggingface_hub
hf download cyankiwi/Qwen3.8-27B-AWQ-INT4

How to use it: The part of the card available here shows no run command, so follow your runtime's own instructions. Thinking mode is on by default. You can turn it off for a single request, tune its depth, or keep reasoning from earlier messages.

Tool | What it does
reasoning_effort | Tunes how deep the reasoning goes
preserve_thinking | Keeps reasoning context from earlier messages
Thinking | Reasons before it answers unless you switch it off for one call

Example: You serve the model with vLLM and send an image along with a question about it. Thinking mode is on by default, so the model reasons first, and you can switch that off for a single request.

For newcomers and experts: Think of it as a large assistant that has been compressed so it fits a smaller machine, and that can look at pictures and videos. A first small try is to serve it in a runtime you know and ask one short question.

AWQ means activation-aware weight quantization, which uses calibration data to decide how to treat the weights. This build stores them at 4 bits. The trade-off is lower memory against some change from the original weights, and the card gives no accuracy comparison.

Download the quantized weights
Load them in a supported runtime
Send text, images or video
Thinking runs unless you disable it
Read the answer

Good to know: The card lists these properties of the build.

Title: This build at a glance
Property | Value
Base model | Qwen3.8-27B
Quantization | AWQ INT4
Model size | 21.02 GB
Input | Text, images and video

• Parameters: 27.8B in the Hugging Face record
• Other versions: other people have published 2 quantized versions
• Accuracy: the card gives no comparison against the original weights
• License: Apache 2.0
• Cost to run: none named, and the card points to a hosted option that is not yet available
• Data it reaches: the files you download and the text, images and video you send
• Not for: matching the unquantized model exactly

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 27.8B
• Weights: about 111 GB in I32
• Formats: Safetensors
• Downloads in the last 30 days: 1,056,172
• Likes: 135
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 2
• Tags: transformers, safetensors, qwen3_5, image-text-to-text, conversational, compressed-tensors

Links

• Model: https://huggingface.co/cyankiwi/Qwen3.8-27B-AWQ-INT4
• Files: https://huggingface.co/cyankiwi/Qwen3.8-27B-AWQ-INT4/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download cyankiwi/Qwen3.8-27B-AWQ-INT4

Then follow the usage in its model card. Nothing in this kit runs by itself.