Qwen3.8-27B-QUASAR-NVFP4

Qwen3.8-27B-QUASAR-NVFP4

Runs Qwen3.8-27B in 4-bit NVFP4 at 19.7 GB, trained with the QUASAR method to stay close to the full BF16 model. For people serving it with vLLM on an NVIDIA Blackwell GPU.

Made by QUASAR-QAT on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a 4-bit NVFP4 version of Qwen3.8-27B, made with QUASAR, a quantization-aware training method. The method teaches the 4-bit weights to imitate an unchanged BF16 copy, and what you get is a regular NVFP4 checkpoint. All 496 transformer linear layers are 4-bit, and the whole model takes 19.7 GB.

Who it suits: Teams that serve a 27B model on one NVIDIA Blackwell GPU and want to keep quality close to the full-precision version.

Industry | Who uses it | How they use it | Example
AI infrastructure | ML engineers | Serve a 27B model on a single Blackwell card | An OpenAI-style endpoint backed by vLLM
Software | Developers | Run a reasoning and tool-calling model locally | An agent that calls tools through the server
Research | Quantization researchers | Compare QUASAR with other NVFP4 builds | Check GPQA-Diamond and AIME'26 scores side by side
Enterprise IT | Platform teams | Cut the memory footprint of a 27B model | 19.7 GB instead of 55.6 GB for BF16

Skip it if you have no NVIDIA GPU with FP4 support, since the card requires Blackwell.

Where to use it:

• vLLM, which serves the model with no conversion step
• NVIDIA Blackwell GPUs, compute capability 10.0 or higher
• A consumer card with 32 GB of memory, such as the RTX 5090, once you cut the context length

Use case | Who | What they get back
Serve a chat endpoint | ML engineers | A vLLM server that answers requests with reasoning and tool calls
Save GPU memory | Platform teams | The model in 19.7 GB instead of 55.6 GB
Long-context work | Developers | A context of up to 262144 tokens on larger cards

How to set it up: You need an NVIDIA GPU with FP4 support, Python with pip and a recent vLLM. The card names vLLM 0.27 or newer.

• An NVIDIA Blackwell GPU
• Python and pip
• vLLM 0.27 or newer

1. Install vLLM.
2. Start the server with the command below, which downloads the model on first use.
3. On a 32 GB card, lower the context length to 65536.

pip install "vllm>=0.27"

vllm serve QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --max-num-seqs 512 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

How to use it: Start the server with the command above and send chat requests to it like any OpenAI-style endpoint. The parsers return tool calls and reasoning in standard fields. On a 32 GB card, shorten the context.

--max-model-len 65536

Tool | What it does
vllm serve | Starts the model server
--max-model-len | Sets the context length in tokens
--gpu-memory-utilization | Sets how much GPU memory the server may use
--speculative-config | Turns on speculative decoding with two draft tokens
--tool-call-parser | Reads the model's tool calls
--reasoning-parser | Separates the model's reasoning from its answer

Example: You start the server and ask the model a hard multi-step question through its chat endpoint. It reasons first and answers, and tool calls arrive in the parsed fields if you offered tools.

For newcomers and experts: If you are new to this, you need a recent NVIDIA card and vLLM, and the first run downloads about 19.7 GB. Start with the shorter context setting.

If you know the field, QUASAR is quantization-aware distillation against a frozen BF16 teacher for one epoch, with batch size 32, learning rate 1e-6 and 2446 steps. It quantizes attention and gated delta-net layers too, and exports standard NVFP4 with no custom inference path.

Frozen BF16 model as teacher
Train NVFP4 weights against it
Export standard NVFP4 weights
Serve directly with vLLM

Good to know: The card compares this build with two other public NVFP4 builds under one setup. Its GPQA-Diamond score is 90.91 against 91.41 for the BF16 original, and AIME'26 is 100.0 for both.

Title: Model size in GB
QUASAR NVFP4: 19.7 GB
Unsloth NVFP4: 23.4 GB
Inferact NVFP4: 26.4 GB
BF16 original: 55.6 GB

• Quantized layers: 496 of 496 for this build, against 168 and 304 for the other two compared
• The scores are the maker's own evaluation
• License: Apache 2.0
• Cost to run: none named, but you need an NVIDIA Blackwell GPU
• Data it reaches: the files vLLM downloads, plus the requests you send to your own server
• Not for: GPUs without FP4 support

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 27.8B
• Weights: about 27.8 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 119,414
• Likes: 164
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Quantized versions: 12
• Fine-tunes: 3
• Tags: transformers, safetensors, qwen3_5, image-text-to-text, qwen3.8, nvfp4, fp4, w4a4, 4bit, quantization, quantized, quantization-aware-training

Links

• Model: https://huggingface.co/QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4
• Files: https://huggingface.co/QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4

Then follow the usage in its model card. Nothing in this kit runs by itself.