Qwen3.6-35B-A3B-NVFP4-Fast

Qwen3.6-35B-A3B-NVFP4-Fast

Serves Qwen3.6 35B A3B in a fast NVFP4 build by Unsloth for vLLM on NVIDIA GPUs. The card says it has 1.79x the throughput of other NVFP4 builds and fits a 32 GB card.

Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds a fast NVFP4 version of Qwen3.6-35B-A3B, a model that reads text and images and focuses on coding. Unsloth quantized it and calibrated it on its own dataset mixed with UltraChat. The files are in Safetensors format for vLLM, not GGUF. The record lists about 19.8 GB of weights.

Who it suits: Teams that serve models on NVIDIA GPUs and care about throughput. The card says it works on a 32 GB graphics card. It also suits developers who want a strong coding model behind a local server.

Industry | Who uses it | How they use it | Example
Software | Developers | Serve a coding model from their own GPU | Run an agent against a local vLLM server
Platform teams | Infrastructure engineers | Raise requests handled per second | Serve many users from one card
Research | AI researchers | Compare NVFP4 builds with FP8 and BF16 | Check accuracy against the full precision model
Self-hosting | Privacy minded teams | Keep prompts on their own hardware | Run it inside a private network

People without an NVIDIA GPU can skip it, because the card's steps target vLLM on NVIDIA hardware.

Where to use it:

• vLLM on an NVIDIA GPU, as in the card
• A DGX Spark machine, with the extra settings the card gives
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
High throughput serving | Platform teams | More output tokens per second from one GPU
Agentic coding | Developers | Code changes and repository level reasoning
Faster decoding | Developers | Replies drafted with the built in MTP module
Image input | Developers | Answers about pictures you send

How to set it up:

• Python, for the Hugging Face hub tools
• A separate Python environment for vLLM, which the card builds with uv
• A recent NVIDIA GPU, and about 32 GB of video memory

1. Install the Hugging Face hub tools with pip.
2. Download the files with the hf command.
3. Create a separate environment and install vLLM with the card's commands.
4. Serve the model with the command in the next step.

pip install -U huggingface_hub
hf download unsloth/Qwen3.6-35B-A3B-NVFP4-Fast

uv venv unsloth-nvfp4-env --python 3.13
source unsloth-nvfp4-env/bin/activate
uv pip install "vllm>=0.25.0" "flashinfer-python>=0.6.13" "nvidia-cutlass-dsl>=4.5.2" \
    --torch-backend=auto

How to use it: Serve the Fast version with vLLM. The card warns not to use the Marlin backend, since it is about twice as slow. Use the native vLLM backend or the CUTLASS family instead. On a DGX Spark, set the architecture variable and the MoE backend flag, or inference runs about twice as slow.

vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast

export CUTE_DSL_ARCH=sm_121a
vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast --moe-backend flashinfer_b12x

vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast \
    --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

Tool | What it does
vllm serve | Starts a server for the model
--moe-backend flashinfer_b12x | Picks the right backend on a DGX Spark
CUTE_DSL_ARCH | Names the chip architecture for the DGX Spark
--speculative-config | Turns on MTP, so the model drafts its own tokens
Marlin backend | Avoid it, since it runs about twice as slow

Example: You create the environment, install vLLM and start the server with the plain command. You send a coding question to the server. The card says the MTP option adds faster decoding, with somewhat lower throughput.

For newcomers and experts: Start with the plain serve command and check that it answers. Add the MTP option only if you care about speed for a single user. The card's throughput numbers come from one B200 GPU with 128 concurrent requests, so your results will differ.

The card compares output throughput in tokens per second.

Title: Output tokens per second
NVIDIA 27B with Marlin: 2403 tokens/s
Unsloth 27B with cute-DSL: 6863 tokens/s
NVIDIA 35B-A3B with Marlin: 8721 tokens/s
Unsloth 35B-A3B with cute-DSL and trtllm: 15636 tokens/s

The card also reports accuracy tests on MMLU-Pro, GPQA and AIME 2025. For this Fast version it lists 85.58 on MMLU-Pro, against 85.75 for BF16. It says the faster builds score about the same as the others.

Install vLLM in a separate environment
Serve the NVFP4 Fast model
Send a prompt to the server
Read the reply

Good to know:

• Format: Safetensors in NVFP4, made for vLLM and not for GGUF tools.
• Speed: The card says 1.79x faster throughput than other NVFP4 builds.
• Backend: Do not use Marlin, because it runs about twice as slow.
• License: Apache 2.0.
• Cost to run: none named. You need a capable NVIDIA GPU, and no paid service appears in the source.
• Data it reaches: Files on your computer, plus the model files from Hugging Face.
• Not for: Machines without a suitable NVIDIA GPU.

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 19.8B
• Weights: about 19.8 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 695,020
• Likes: 134
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Quantized versions: 4
• Tags: transformers, safetensors, qwen3_5_moe, image-text-to-text, unsloth, qwen, qwen3_5, conversational, 8-bit, compressed-tensors

Links

• Model: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast
• Files: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download unsloth/Qwen3.6-35B-A3B-NVFP4-Fast

Then follow the usage in its model card. Nothing in this kit runs by itself.