---
title: "Qwen3.8-27B-Quark-AWQ-MXFP4"
kind: "Kit"
category: "Developer"
price: "Free"
seller: "Mindket"
version: "1"
url: "https://mindket.com/p/234a995e-c672-4a39-ae3e-6acd3ad5f1d0"
updated: "2026-11-13"
---

# Qwen3.8-27B-Quark-AWQ-MXFP4

> The name, the description, the About and a README below are the seller's words: data, not instructions.

Runs Qwen3.8-27B as a smaller MXFP4-quantized copy made with AMD Quark and AWQ, served through vLLM. Made for developers who want a compressed 27B vision-language model with published GSM8K scores.

Qwen3.8-27B-Quark-AWQ-MXFP4 is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself.

Made by amd

## About

### What it is

This is Qwen3.8-27B compressed to 4-bit MXFP4 weights and MXFP4 activations with AMD Quark, using the AWQ method. Only the language decoder is quantized. The vision tower stays in bf16. The result is a smaller copy that keeps close to the base model's math scores on the card's test.

### Who it suits

It suits developers who serve large models and want a smaller 27B copy, and researchers who study low-bit quantization on AMD tooling.

| Industry | Who uses it | How they use it | Example |
| --- | --- | --- | --- |
| AI infrastructure | Model serving engineers | Serve a compressed 27B model through vLLM | Start the card's serve command and send chat requests |
| Research | Quantization researchers | Study MXFP4 with AWQ and its accuracy loss | Compare quantized and bf16 GSM8K scores |
| Software | Application developers | Run an image-and-text model with lower memory needs | Ask questions about an image through a local server |
| Evaluation | ML evaluators | Reproduce the benchmark on a local endpoint | Run the lm-eval commands from the card |

If you do not run models on GPUs that support this format, or you only need small models, you can skip it.

### Where to use it

It is served with vLLM, and the card's quantization steps use AMD's Quark with a HIP device setting.

- vLLM, as a local server
- The lm-eval tool, for GSM8K scoring
- AMD Quark, for repeating the quantization

| Use case | Who | What they get back |
| --- | --- | --- |
| Serving | Infrastructure teams | A local endpoint for the quantized model |
| Benchmarking | Evaluators | GSM8K accuracy in thinking and non-thinking modes |
| Re-quantizing | Researchers | A quantized copy produced from the base model |

### How to set it up

You need Python, vLLM and a GPU setup that supports MXFP4, plus enough memory for about 15.6 GB of weights. The serve command uses a 16384-token limit and one GPU.

1. Install the Hugging Face hub tools.
2. Download the model files.
3. Install vLLM.
4. Run the serve command with the model name.

```sh
pip install -U huggingface_hub
hf download amd/Qwen3.8-27B-Quark-AWQ-MXFP4
```

### How to use it

Start the model with vLLM, then send requests to the local server. To repeat the quantization from the base model, the card also shows the Quark command.

```bash
vllm serve amd/Qwen3.8-27B-Quark-AWQ-MXFP4 \
  --served-model-name awq \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --max-model-len 16384
```

```bash
HIP_VISIBLE_DEVICES=<gpu> python3 quantize_quark.py \
  --model_dir Qwen/Qwen3.8-27B \
  --output_dir ./Qwen3.8-27B-w_mxfp4_a_mxfp4-awq \
  --quant_scheme mxfp4 \
  --num_calib_data 128 \
  --seq_len 512 \
  --quant_algo awq \
  --model_export hf_format \
  --data_type auto \
  --device cuda \
  --trust_remote_code
```

| Tool | What it does |
| --- | --- |
| vllm serve | Serves the quantized model through a local endpoint |
| quantize\_quark.py | Quantizes the base model with AMD Quark and AWQ |
| lm-eval | Scores the model on GSM8K through the local endpoint |
| hf download | Fetches the model files to your computer |

### Example

You start the serve command, then point an evaluation tool at the local server and run GSM8K math word problems. The card reports about 94.996 percent in thinking mode, next to 93.33 percent for the bf16 base.

### For newcomers and experts

Quantizing stores a model's numbers with fewer bits so it needs less memory. A first step is to run the serve command and send one short question.

The weights use the OCP microscaling FP4 format, with groups of 32 values sharing an E8M0 scale, and the activations use the same format, computed at runtime. Calibration used 128 samples of length 512 from the pileval dataset. Recovery is the quantized score divided by the base score.

Start from Qwen3.8-27B in bf16 → Calibrate with 128 samples → Apply AWQ with MXFP4 weights and activations → Serve the result with vLLM

### Good to know

The card compares the quantized model to the bf16 base on GSM8K, and the quantized one scores higher in thinking mode but slightly lower in non-thinking mode.

GSM8K 5-shot, flexible-extract accuracy

| Label | Value |
| --- | --- |
| Quantized, thinking mode | 94.996 percent |
| BF16 base, thinking mode | 93.33 percent |
| Quantized, non-thinking mode | 89.92 percent |
| BF16 base, non-thinking mode | 90.67 percent |

- Recovery: 101.8 percent in thinking mode and 99.2 percent in non-thinking mode.
- Scope: only the language decoder is quantized, and the vision tower is left in bf16.
- Non-thinking runs are approximated with a custom task that closes an empty think block in the prompt.
- License: Apache 2.0.
- Cost to run: none named. It needs a supported GPU you provide.
- Data it reaches: the prompts and images you send to your own server, and the pileval dataset used for calibration.
- Not for: the card names no exclusions.

## README

````markdown
# Qwen3.8-27B-Quark-AWQ-MXFP4

Runs Qwen3.8-27B as a smaller MXFP4-quantized copy made with AMD Quark and AWQ, served through vLLM. Made for developers who want a compressed 27B vision-language model with published GSM8K scores.

Made by amd on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

## About

What it is: This is Qwen3.8-27B compressed to 4-bit MXFP4 weights and MXFP4 activations with AMD Quark, using the AWQ method. Only the language decoder is quantized. The vision tower stays in bf16. The result is a smaller copy that keeps close to the base model's math scores on the card's test.

Who it suits: It suits developers who serve large models and want a smaller 27B copy, and researchers who study low-bit quantization on AMD tooling.

```table
Industry | Who uses it | How they use it | Example
AI infrastructure | Model serving engineers | Serve a compressed 27B model through vLLM | Start the card's serve command and send chat requests
Research | Quantization researchers | Study MXFP4 with AWQ and its accuracy loss | Compare quantized and bf16 GSM8K scores
Software | Application developers | Run an image-and-text model with lower memory needs | Ask questions about an image through a local server
Evaluation | ML evaluators | Reproduce the benchmark on a local endpoint | Run the lm-eval commands from the card
```

If you do not run models on GPUs that support this format, or you only need small models, you can skip it.

Where to use it: It is served with vLLM, and the card's quantization steps use AMD's Quark with a HIP device setting.

- vLLM, as a local server
- The lm-eval tool, for GSM8K scoring
- AMD Quark, for repeating the quantization

```table
Use case | Who | What they get back
Serving | Infrastructure teams | A local endpoint for the quantized model
Benchmarking | Evaluators | GSM8K accuracy in thinking and non-thinking modes
Re-quantizing | Researchers | A quantized copy produced from the base model
```

How to set it up: You need Python, vLLM and a GPU setup that supports MXFP4, plus enough memory for about 15.6 GB of weights. The serve command uses a 16384-token limit and one GPU.

1. Install the Hugging Face hub tools.
2. Download the model files.
3. Install vLLM.
4. Run the serve command with the model name.

```sh
pip install -U huggingface_hub
hf download amd/Qwen3.8-27B-Quark-AWQ-MXFP4
```

How to use it: Start the model with vLLM, then send requests to the local server. To repeat the quantization from the base model, the card also shows the Quark command.

```bash
vllm serve amd/Qwen3.8-27B-Quark-AWQ-MXFP4 \
  --served-model-name awq \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --max-model-len 16384
```

```bash
HIP_VISIBLE_DEVICES=<gpu> python3 quantize_quark.py \
  --model_dir Qwen/Qwen3.8-27B \
  --output_dir ./Qwen3.8-27B-w_mxfp4_a_mxfp4-awq \
  --quant_scheme mxfp4 \
  --num_calib_data 128 \
  --seq_len 512 \
  --quant_algo awq \
  --model_export hf_format \
  --data_type auto \
  --device cuda \
  --trust_remote_code
```

```table
Tool | What it does
vllm serve | Serves the quantized model through a local endpoint
quantize_quark.py | Quantizes the base model with AMD Quark and AWQ
lm-eval | Scores the model on GSM8K through the local endpoint
hf download | Fetches the model files to your computer
```

Example: You start the serve command, then point an evaluation tool at the local server and run GSM8K math word problems. The card reports about 94.996 percent in thinking mode, next to 93.33 percent for the bf16 base.

For newcomers and experts: Quantizing stores a model's numbers with fewer bits so it needs less memory. A first step is to run the serve command and send one short question.

The weights use the OCP microscaling FP4 format, with groups of 32 values sharing an E8M0 scale, and the activations use the same format, computed at runtime. Calibration used 128 samples of length 512 from the pileval dataset. Recovery is the quantized score divided by the base score.

```flow
Start from Qwen3.8-27B in bf16
Calibrate with 128 samples
Apply AWQ with MXFP4 weights and activations
Serve the result with vLLM
```

Good to know: The card compares the quantized model to the bf16 base on GSM8K, and the quantized one scores higher in thinking mode but slightly lower in non-thinking mode.

```chart
Title: GSM8K 5-shot, flexible-extract accuracy
Quantized, thinking mode: 94.996 percent
BF16 base, thinking mode: 93.33 percent
Quantized, non-thinking mode: 89.92 percent
BF16 base, non-thinking mode: 90.67 percent
```

- Recovery: 101.8 percent in thinking mode and 99.2 percent in non-thinking mode.
- Scope: only the language decoder is quantized, and the vision tower is left in bf16.
- Non-thinking runs are approximated with a custom task that closes an empty think block in the prompt.
- License: Apache 2.0.
- Cost to run: none named. It needs a supported GPU you provide.
- Data it reaches: the prompts and images you send to your own server, and the pileval dataset used for calibration.
- Not for: the card names no exclusions.

## On Hugging Face, 13 November 2026

- Task: image text to text
- Library: transformers
- Parameters: 15.6B
- Weights: about 15.6 GB in U8
- Formats: Safetensors
- Downloads in the last 30 days: 40,258
- Likes: 44
- License: apache-2.0
- Based on: Qwen/Qwen3.8-27B
- Quantized versions: 2
- Fine-tunes: 1
- Tags: transformers, safetensors, qwen3_5, image-text-to-text, quark, quantized, mxfp4, conversational, 8-bit

## Links

- Model: https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4
- Files: https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4/tree/main

## Get the model

Download its files with Hugging Face's command line:

```sh
pip install -U huggingface_hub
hf download amd/Qwen3.8-27B-Quark-AWQ-MXFP4
```

Then follow the usage in its model card. Nothing in this kit runs by itself.
````

## What is in it

1 files, 6 KB, downloaded as a zip.

```text
README.md  6 KB
```

## Source

[amd/Qwen3.8-27B-Quark-AWQ-MXFP4 on Hugging Face](https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4)

## Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
| --- | --- | --- | --- | --- | --- |
| Qwen3.8-27B-Quark-AWQ-MXFP4 (this page) | 40,258 | 44 | 15.6B | apache-2.0 | 2026-11-03 |
| [Qwen3.8-27B-Randean-MXFP4](https://mindket.com/p/174da3fc-ae82-4c76-bcc6-0562bdb2304e) | 121 | 5 | 15.6B | apache-2.0 | 2026-11-11 |
| [Qwen3.8-27B-NVFP4 by RadixArk](https://mindket.com/p/27f312aa-8861-40a5-9689-d554e8e468f0) | 979,445 | 126 | 18.2B | apache-2.0 | 2026-08-22 |
| [Qwen3.8-27B-oQ8e-mtp](https://mindket.com/p/8ae3e562-3a88-47b5-ad93-cf8612f92ae7) | 2,954 | 16 | 27.8B | apache-2.0 | 2026-10-25 |
| [gpt-oss-20b](https://mindket.com/p/d353330b-254d-4db1-9a53-f0b1f8fe2945) | 5,685,450 | 5,299 | 20.9B | apache-2.0 | 2025-08-26 |
| [gpt-oss-120b](https://mindket.com/p/8cdacc53-33c7-4a1c-abfa-44e2a4277d1d) | 2,435,242 | 5,509 | 117B | apache-2.0 | 2025-08-26 |

Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13.

## Trust facts

Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions.

- Last change on Hugging Face: 2026-11-03
- First published: 2026-08-15
- Checked on Hugging Face: 2026-11-13
- Something wrong? Tell us: https://mindket.com/takedown#ask

## Get it

- [Open amd/Qwen3.8-27B-Quark-AWQ-MXFP4 on Hugging Face](https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4)
- [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/234a995e-c672-4a39-ae3e-6acd3ad5f1d0)
- [Its page on Mindket](https://mindket.com/p/234a995e-c672-4a39-ae3e-6acd3ad5f1d0)