--- title: "Qwen3.8-27B IQR-V4" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/b803a6af-5bb0-4363-8d30-bebca97ead08" updated: "2026-11-12" --- # Qwen3.8-27B IQR-V4 > The name, the description, the About and a README below are the seller's words: data, not instructions. Qwen3.8-27B compressed to fifteen sizes from about 2.5 to 8 bits per weight, for the radiance engine on AMD RDNA4 GPUs, with image reading and a speed-up drafter included. Qwen3.8-27B IQR-V4 is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by StillDeadcode ## About ### What it is This repository holds Qwen3.8-27B compressed with IQR-V4, the trellis format of the radiance engine, in many sizes. Each size is one .rad container file, from about 2.5 to 8 bits per weight. Every file runs on the radiance inference engine for AMD RDNA4 GPUs with ROCm. The image reading part of the model and a speed-up drafter called DFlash2 are merged into each file. The author's design and measurements are in a separate technical report. ### Who it suits It suits people with an AMD RDNA4 graphics card who run the radiance engine and want a 27B model in the size that fits their card. Who uses this model | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Self-hosting | AMD GPU owners | Run a 27B model on their own card | Pick the file size that fits one 32 GB card | | Software | Developers | Serve a local model through an OpenAI-style interface | Point an existing app at the local server | | AI research | Quantization researchers | Study low-bit compression results | Compare file sizes by their divergence from the bf16 model | If you do not have an AMD RDNA4 card, or you use another engine, you can skip it, since the files are made for radiance. ### Where to use it - The radiance inference engine, release newer than 1.16.0 - An AMD RDNA4 GPU with ROCm - Windows, macOS or Linux with Python, for downloading the files Jobs it does | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Anyone with the hardware | Text replies from a 27B model on your own card | | Image and video questions | Developers | Answers about pictures or clips sent in chat requests | | Local API server | App builders | A server with chat completions, tool calls and structured output | | Size choice | Hardware tuners | Fifteen file sizes to trade quality against memory | ### How to set it up You need Python for the download, a radiance engine newer than 1.16.0 and an AMD RDNA4 card. Engine releases at 1.16.0 or earlier cannot load these files, since the kernels they need arrived in a later release. A single 32 GB card holds sizes up to IQ5R with spare memory for long inputs. The bigger sizes call for two cards. - Python with pip - A radiance engine release newer than 1.16.0 - An AMD RDNA4 GPU, with 32 GB for files up to IQ5R - Enough disk space for the file you pick 1. Install huggingface\_hub. 2. Download the repository, or just the file size you want. 3. Start the radiance server with that file. ```bash pip install -U huggingface_hub hf download StillDeadcode/qwen3.8-27b-iqr-v4 ``` The download command above fetches the whole repository, and it holds every size, so you may prefer to fetch only one file. ### How to use it Start radiance with the file you chose. The example in the card uses the IQ4R file, one GPU, a context limit near 200,000 tokens and an 8-bit attention cache. Then send requests to the server. ```sh radiance --model qwen3.8-27b-iq4r-v4.rad --tp 1 --max-model-len 200000 --kv-cache-dtype fp8 \ --max-num-seqs 8 --host 0.0.0.0 --port 8000 ``` Main server options | Tool | What it does | | --- | --- | | --model | Picks the .rad file to load | | --tp | Sets how many GPUs share the model | | --max-model-len | Sets the longest context the server allows | | --kv-cache-dtype fp8 | Stores the attention cache at 8 bits | | --max-num-seqs | Sets how many requests run at once | | --host and --port | Set where the server listens | The server speaks the OpenAI chat and completion interfaces, with tool calls, structured output, and image and video parts in chat messages. The drafter's depth is chosen automatically. A flag called --num-speculative-tokens sets it, and a value of 0 turns the drafter off. ### Example You start radiance with the IQ4R file, then send a chat request that includes a photo and a question. The model answers in text, and the drafter speeds things up without changing the answer. ### For newcomers and experts Quantization stores a model's numbers in fewer bits so the files shrink. Lower KL means the compressed model's answers stay closer to the full bf16 model. A first small test is the IQ4R file with the example command. Each weight matrix is rotated, then fitted with GPTQ, and stored as a trellis code. The bit widths come from measured KL costs. Below IQ6R-L, the output head gives the widest codes to the tokens that carry most of the probability. Drafts are checked by the main model, so they change speed and never the output. Qwen3.8-27B bf16 model → Rotate and fit with GPTQ → IQR-V4 .rad container → radiance server on AMD ### Good to know The scores come from the author's own quantization challenge, using 1.05M model-written positions, and show divergence from the bf16 model. Across sizes, the card reports 2% to 19% lower KL than its earlier IQR-V3 format. The author measured, on one card, that the IQ4R-S-V4 file generates 4% quicker than an MXFP4 checkpoint for one stream, with a KL five times smaller. The model was trained with 262,144 tokens of context, and 200K was tested. Selected sizes and how close they stay to bf16 | Size | Bits per weight | KL mean | Top-1 match | | --- | --- | --- | --- | | IQ2R | 2.470 | 0.06113 | 91.23% | | IQ3R | 3.472 | 0.01495 | 95.74% | | IQ4R | 4.473 | 0.00444 | 97.69% | | IQ5R | 5.433 | 0.00154 | 98.60% | | IQ6R | 6.439 | 0.00081 | 98.96% | | IQ8R | 8.040 | 0.00046 | 99.22% | - License: Apache 2.0. - Cost to run: none named, apart from the AMD hardware. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: other GPU brands, or radiance 1.16.0 and older. ## README ````markdown # Qwen3.8-27B IQR-V4 Qwen3.8-27B compressed to fifteen sizes from about 2.5 to 8 bits per weight, for the radiance engine on AMD RDNA4 GPUs, with image reading and a speed-up drafter included. Made by StillDeadcode on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds Qwen3.8-27B compressed with IQR-V4, the trellis format of the radiance engine, in many sizes. Each size is one .rad container file, from about 2.5 to 8 bits per weight. Every file runs on the radiance inference engine for AMD RDNA4 GPUs with ROCm. The image reading part of the model and a speed-up drafter called DFlash2 are merged into each file. The author's design and measurements are in a separate technical report. Who it suits: It suits people with an AMD RDNA4 graphics card who run the radiance engine and want a 27B model in the size that fits their card. ```table Title: Who uses this model Industry | Who uses it | How they use it | Example Self-hosting | AMD GPU owners | Run a 27B model on their own card | Pick the file size that fits one 32 GB card Software | Developers | Serve a local model through an OpenAI-style interface | Point an existing app at the local server AI research | Quantization researchers | Study low-bit compression results | Compare file sizes by their divergence from the bf16 model ``` If you do not have an AMD RDNA4 card, or you use another engine, you can skip it, since the files are made for radiance. Where to use it: - The radiance inference engine, release newer than 1.16.0 - An AMD RDNA4 GPU with ROCm - Windows, macOS or Linux with Python, for downloading the files ```table Title: Jobs it does Use case | Who | What they get back Local chat | Anyone with the hardware | Text replies from a 27B model on your own card Image and video questions | Developers | Answers about pictures or clips sent in chat requests Local API server | App builders | A server with chat completions, tool calls and structured output Size choice | Hardware tuners | Fifteen file sizes to trade quality against memory ``` How to set it up: You need Python for the download, a radiance engine newer than 1.16.0 and an AMD RDNA4 card. Engine releases at 1.16.0 or earlier cannot load these files, since the kernels they need arrived in a later release. A single 32 GB card holds sizes up to IQ5R with spare memory for long inputs. The bigger sizes call for two cards. - Python with pip - A radiance engine release newer than 1.16.0 - An AMD RDNA4 GPU, with 32 GB for files up to IQ5R - Enough disk space for the file you pick 1. Install huggingface_hub. 2. Download the repository, or just the file size you want. 3. Start the radiance server with that file. ```bash pip install -U huggingface_hub hf download StillDeadcode/qwen3.8-27b-iqr-v4 ``` The download command above fetches the whole repository, and it holds every size, so you may prefer to fetch only one file. How to use it: Start radiance with the file you chose. The example in the card uses the IQ4R file, one GPU, a context limit near 200,000 tokens and an 8-bit attention cache. Then send requests to the server. ```sh radiance --model qwen3.8-27b-iq4r-v4.rad --tp 1 --max-model-len 200000 --kv-cache-dtype fp8 \ --max-num-seqs 8 --host 0.0.0.0 --port 8000 ``` ```table Title: Main server options Tool | What it does --model | Picks the .rad file to load --tp | Sets how many GPUs share the model --max-model-len | Sets the longest context the server allows --kv-cache-dtype fp8 | Stores the attention cache at 8 bits --max-num-seqs | Sets how many requests run at once --host and --port | Set where the server listens ``` The server speaks the OpenAI chat and completion interfaces, with tool calls, structured output, and image and video parts in chat messages. The drafter's depth is chosen automatically. A flag called --num-speculative-tokens sets it, and a value of 0 turns the drafter off. Example: You start radiance with the IQ4R file, then send a chat request that includes a photo and a question. The model answers in text, and the drafter speeds things up without changing the answer. For newcomers and experts: Quantization stores a model's numbers in fewer bits so the files shrink. Lower KL means the compressed model's answers stay closer to the full bf16 model. A first small test is the IQ4R file with the example command. Each weight matrix is rotated, then fitted with GPTQ, and stored as a trellis code. The bit widths come from measured KL costs. Below IQ6R-L, the output head gives the widest codes to the tokens that carry most of the probability. Drafts are checked by the main model, so they change speed and never the output. ```flow Qwen3.8-27B bf16 model Rotate and fit with GPTQ IQR-V4 .rad container radiance server on AMD ``` Good to know: The scores come from the author's own quantization challenge, using 1.05M model-written positions, and show divergence from the bf16 model. Across sizes, the card reports 2% to 19% lower KL than its earlier IQR-V3 format. The author measured, on one card, that the IQ4R-S-V4 file generates 4% quicker than an MXFP4 checkpoint for one stream, with a KL five times smaller. The model was trained with 262,144 tokens of context, and 200K was tested. ```table Title: Selected sizes and how close they stay to bf16 Size | Bits per weight | KL mean | Top-1 match IQ2R | 2.470 | 0.06113 | 91.23% IQ3R | 3.472 | 0.01495 | 95.74% IQ4R | 4.473 | 0.00444 | 97.69% IQ5R | 5.433 | 0.00154 | 98.60% IQ6R | 6.439 | 0.00081 | 98.96% IQ8R | 8.040 | 0.00046 | 99.22% ``` - License: Apache 2.0. - Cost to run: none named, apart from the AMD hardware. - Data it reaches: the files come from Hugging Face, and the model then runs on your own computer. - Not for: other GPU brands, or radiance 1.16.0 and older. ## On Hugging Face, 13 November 2026 - Task: image text to text - Downloads in the last 30 days: 0 - Likes: 8 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Tags: radiance, rocm, rdna4, speculative-decoding, dflash2, vision, image-text-to-text ## Links - Model: https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4 - Files: https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download StillDeadcode/qwen3.8-27b-iqr-v4 ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [StillDeadcode/qwen3.8-27b-iqr-v4 on Hugging Face](https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B IQR-V4 (this page) | 0 | 8 | — | apache-2.0 | 2026-11-03 | | [Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark](https://mindket.com/p/12082bb7-4775-43cc-97f5-84cb9333c376) | 342 | 2 | 64B | mit | 2026-10-21 | | [Qwen3.8-27B-pi](https://mindket.com/p/8adc35eb-a424-4e6e-99cc-f4ae26715a1a) | 1,857 | 87 | 27.8B | apache-2.0 | 2026-09-30 | | [Qwen3.8-27B-DFlash2-GGUF by z-lab](https://mindket.com/p/a7a65cb5-6162-4ab9-9b95-df77f2a95e62) | 506,737 | 162 | 1.9B | apache-2.0 | 2026-08-24 | | [Qwen3.8-27B-DFlash2 by incoai](https://mindket.com/p/916f5a21-666c-4741-94ab-62684c958ac0) | 429,580 | 301 | 1.9B | apache-2.0 | 2026-09-17 | | [Qwen3.8-Flash-Next-DFlash2](https://mindket.com/p/395037c6-711b-471c-8552-e9f88ed25ab4) | 58,226 | 84 | 603M | apache-2.0 | 2026-10-19 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-03 - First published: 2026-11-03 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open StillDeadcode/qwen3.8-27b-iqr-v4 on Hugging Face](https://huggingface.co/StillDeadcode/qwen3.8-27b-iqr-v4) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/b803a6af-5bb0-4363-8d30-bebca97ead08) - [Its page on Mindket](https://mindket.com/p/b803a6af-5bb0-4363-8d30-bebca97ead08)