Qwen3.8-27B-exl3

Qwen3.8-27B-exl3

EXL3 quantized copies of Qwen3.8-27B in many bits-per-weight sizes, from 1.40 to 6.00. For people running the 27B model locally with ExLlamaV3 on limited GPU memory.

Made by turboderp on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3.8-27B-exl3 is a set of EXL3 quantizations of the Qwen3.8-27B model, made by turboderp. Each size lives in its own branch of the repository. The set includes self-calibrated quants from 1.40 to 6.00 bits per weight and plain quants from 2.00 to 6.00.

Who it suits: People who run large language models on their own GPUs and need to fit a 27B model into a given amount of memory. The range of sizes lets you trade memory against quality.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists with one GPU | Fit a 27B model into limited video memory | Pick a lower bits-per-weight variant and load it
Software | Developers who self-host models | Serve Qwen3.8-27B from their own hardware | Choose a mid-range quant for an internal assistant
Research | Quantization researchers | Compare plain and self-calibrated quants | Read the calibration and eval traces in the repository
Enthusiasts | Model testers | Try vision-tower variants of the model | Load a V variant with ExLlamaV3 v1.4.4

You can skip it if your tools only read GGUF files or hosted APIs, since this format targets ExLlamaV3.

Where to use it:

• ExLlamaV3, the loader the card names for the vision-tower variants
• Windows, macOS or Linux with Python, for downloading the files
• Hugging Face, which hosts every quant as its own branch

Use case | Who | What they get back
Low-memory run | Single-GPU users | A smaller quant of the 27B model
Higher-quality run | Users with spare memory | A quant nearer 6 bits per weight
Vision input | Multimodal testers | A V variant with a quantized vision tower

How to set it up: You need Python, a way to run EXL3 files and enough disk space for the quant you choose.

• Python on Windows, macOS or Linux
• ExLlamaV3, version 1.4.4 for the vision-tower variants
• A GPU with room for the quant you pick

1. Install Python and the Hugging Face hub tool.
2. Choose a quant from the list on the model page.
3. Download its files, which sit in the branch named for it.
4. Load the folder in ExLlamaV3.

pip install -U huggingface_hub
hf download turboderp/Qwen3.8-27B-exl3

How to use it: The page shows no run script. Pick the variant that matches your memory, download that branch and load it in ExLlamaV3 like any other EXL3 model. Variants marked V include a quantized vision tower.

Tool | What it does
Self-calibrated quants | Sizes from 1.40 to 6.00 bits per weight, labelled H3 to H6
V variants | The same quants with a quantized vision tower, needing ExLlamaV3 v1.4.4
Plain quants | Seven sizes from 2.00 to 6.00 bits per weight
Calibration and eval traces | Files that record how the quants were calibrated and tested

Example: You have a GPU with limited memory, so you download a lower size such as 2.50 bits per weight from the plain list and load it in ExLlamaV3. Charts on the page let you compare the variants before you decide.

For newcomers and experts: Quantizing shrinks a big model by storing its numbers with fewer bits. Fewer bits means a smaller file and less memory use, usually with some loss of quality. A first step is to pick a middle size, such as 4.00 bits per weight, and see whether it fits.

EXL3 is the format that ExLlamaV3 reads. The self-calibrated variants carry settings labelled H3 to H6, and the V variants add a quantized vision tower. The repository includes calibration and eval traces, so you can see how the quants were produced and judged.

Pick a bits-per-weight variant
Download that branch of the repository
Load it in ExLlamaV3
Run Qwen3.8-27B on your GPU

Good to know:

• Each quant is a branch, so check which one you download.
• Variants with a quantized vision tower need ExLlamaV3 v1.4.4 to work properly.
• The page lists the base model as Qwen3.8-27B.
• License: apache-2.0.
• Cost to run: none named, you need a GPU that can hold the quant.
• Data it reaches: the files you download from Hugging Face.
• Not for: the page names no limits.

On Hugging Face, 13 November 2026

• Downloads in the last 30 days: 5,032
• Likes: 203
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 1
• Tags: exl3

Links

• Model: https://huggingface.co/turboderp/Qwen3.8-27B-exl3
• Files: https://huggingface.co/turboderp/Qwen3.8-27B-exl3/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download turboderp/Qwen3.8-27B-exl3

Then follow the usage in its model card. Nothing in this kit runs by itself.