Qwen3.8-27B IQR-V4

Qwen3.8-27B IQR-V4

Qwen3.8-27B compressed to fifteen sizes from about 2.5 to 8 bits per weight, for the radiance engine on AMD RDNA4 GPUs, with image reading and a speed-up drafter included.

Made by StillDeadcode on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds Qwen3.8-27B compressed with IQR-V4, the trellis format of the radiance engine, in many sizes. Each size is one .rad container file, from about 2.5 to 8 bits per weight. Every file runs on the radiance inference engine for AMD RDNA4 GPUs with ROCm. The image reading part of the model and a speed-up drafter called DFlash2 are merged into each file. The author's design and measurements are in a separate technical report.

Who it suits: It suits people with an AMD RDNA4 graphics card who run the radiance engine and want a 27B model in the size that fits their card.

Title: Who uses this model
Industry | Who uses it | How they use it | Example
Self-hosting | AMD GPU owners | Run a 27B model on their own card | Pick the file size that fits one 32 GB card
Software | Developers | Serve a local model through an OpenAI-style interface | Point an existing app at the local server
AI research | Quantization researchers | Study low-bit compression results | Compare file sizes by their divergence from the bf16 model

If you do not have an AMD RDNA4 card, or you use another engine, you can skip it, since the files are made for radiance.

Where to use it:

• The radiance inference engine, release newer than 1.16.0
• An AMD RDNA4 GPU with ROCm
• Windows, macOS or Linux with Python, for downloading the files

Title: Jobs it does
Use case | Who | What they get back
Local chat | Anyone with the hardware | Text replies from a 27B model on your own card
Image and video questions | Developers | Answers about pictures or clips sent in chat requests
Local API server | App builders | A server with chat completions, tool calls and structured output
Size choice | Hardware tuners | Fifteen file sizes to trade quality against memory

How to set it up: You need Python for the download, a radiance engine newer than 1.16.0 and an AMD RDNA4 card. Engine releases at 1.16.0 or earlier cannot load these files, since the kernels they need arrived in a later release. A single 32 GB card holds sizes up to IQ5R with spare memory for long inputs. The bigger sizes call for two cards.

• Python with pip
• A radiance engine release newer than 1.16.0
• An AMD RDNA4 GPU, with 32 GB for files up to IQ5R
• Enough disk space for the file you pick

1. Install huggingface_hub.
2. Download the repository, or just the file size you want.
3. Start the radiance server with that file.

pip install -U huggingface_hub
hf download StillDeadcode/qwen3.8-27b-iqr-v4

The download command above fetches the whole repository, and it holds every size, so you may prefer to fetch only one file.

How to use it: Start radiance with the file you chose. The example in the card uses the IQ4R file, one GPU, a context limit near 200,000 tokens and an 8-bit attention cache. Then send requests to the server.

radiance --model qwen3.8-27b-iq4r-v4.rad --tp 1 --max-model-len 200000 --kv-cache-dtype fp8 \
    --max-num-seqs 8 --host 0.0.0.0 --port 8000

Title: Main server options
Tool | What it does
--model | Picks the .rad file to load
--tp | Sets how many GPUs share the model
--max-model-len | Sets the longest context the server allows
--kv-cache-dtype fp8 | Stores the attention cache at 8 bits
--max-num-seqs | Sets how many requests run at once
--host and --port | Set where the server listens

The server speaks the OpenAI chat and completion interfaces, with tool calls, structured output, and image and video parts in chat messages. The drafter's depth is chosen automatically. A flag called --num-speculative-tokens sets it, and a value of 0 turns the drafter off.

Example: You start radiance with the IQ4R file, then send a chat request that includes a photo and a question. The model answers in text, and the drafter speeds things up without changing the answer.

For newcomers and experts: Quantization stores a model's numbers in fewer bits so the files shrink. Lower KL means the compressed model's answers stay closer to the full bf16 model. A first small test is the IQ4R file with the example command.

Each weight matrix is rotated, then fitted with GPTQ, and stored as a trellis code. The bit widths come from measured KL costs. Below IQ6R-L, the output head gives the widest codes to the tokens that carry most of the probability. Drafts are checked by the main model, so they change speed and never the output.

Qwen3.8-27B bf16 model
Rotate and fit with GPTQ
IQR-V4 .rad container
radiance server on AMD

Good to know: The scores come from the author's own quantization challenge, using 1.05M model-written positions, and show divergence from the bf16 model. Across sizes, the card reports 2% to 19% lower KL than its earlier IQR-V3 format. The author measured, on one card, that the IQ4R-S-V4 file generates 4% quicker than an MXFP4 checkpoint for one stream, with a KL five times smaller. The model was trained with 262,144 tokens of context, and 200K was tested.

Title: Selected sizes and how close they stay to bf16
Size | Bits per weight | KL mean | Top-1 match
IQ2R | 2.470 | 0.06113 | 91.23%
IQ3R | 3.472 | 0.01495 | 95.74%
IQ4R | 4.473 | 0.00444 | 97.69%
IQ5R | 5.433 | 0.00154 | 98.60%
IQ6R | 6.439 | 0.00081 | 98.96%
IQ8R | 8.040 | 0.00046 | 99.22%

• License: Apache 2.0.
• Cost to run: none named, apart from the AMD hardware.
• Data it reaches: the files come from Hugging Face, and the model then runs on your own computer.
• Not for: other GPU brands, or radiance 1.16.0 and older.

On Hugging Face, 13 November 2026

• Task: image text to text
• Downloads in the last 30 days: 0
• Likes: 8
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: radiance, rocm, rdna4, speculative-decoding, dflash2, vision, image-text-to-text

Links

• Model: https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4
• Files: https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download StillDeadcode/qwen3.8-27b-iqr-v4

Then follow the usage in its model card. Nothing in this kit runs by itself.