--- title: "Qwen3.6-35B-A3B-NVFP4-Fast" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/dab11469-0f18-43a7-b023-f4a9bfcdea9c" updated: "2026-11-13" --- # Qwen3.6-35B-A3B-NVFP4-Fast > The name, the description, the About and a README below are the seller's words: data, not instructions. Serves Qwen3.6 35B A3B in a fast NVFP4 build by Unsloth for vLLM on NVIDIA GPUs. The card says it has 1.79x the throughput of other NVFP4 builds and fits a 32 GB card. Qwen3.6-35B-A3B-NVFP4-Fast is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by unsloth ## About ### What it is This repository holds a fast NVFP4 version of Qwen3.6-35B-A3B, a model that reads text and images and focuses on coding. Unsloth quantized it and calibrated it on its own dataset mixed with UltraChat. The files are in Safetensors format for vLLM, not GGUF. The record lists about 19.8 GB of weights. ### Who it suits Teams that serve models on NVIDIA GPUs and care about throughput. The card says it works on a 32 GB graphics card. It also suits developers who want a strong coding model behind a local server. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers | Serve a coding model from their own GPU | Run an agent against a local vLLM server | | Platform teams | Infrastructure engineers | Raise requests handled per second | Serve many users from one card | | Research | AI researchers | Compare NVFP4 builds with FP8 and BF16 | Check accuracy against the full precision model | | Self-hosting | Privacy minded teams | Keep prompts on their own hardware | Run it inside a private network | People without an NVIDIA GPU can skip it, because the card's steps target vLLM on NVIDIA hardware. ### Where to use it - vLLM on an NVIDIA GPU, as in the card - A DGX Spark machine, with the extra settings the card gives - Windows, macOS or Linux with Python, for downloading the files | Use case | Who | What they get back | | --- | --- | --- | | High throughput serving | Platform teams | More output tokens per second from one GPU | | Agentic coding | Developers | Code changes and repository level reasoning | | Faster decoding | Developers | Replies drafted with the built in MTP module | | Image input | Developers | Answers about pictures you send | ### How to set it up - Python, for the Hugging Face hub tools - A separate Python environment for vLLM, which the card builds with uv - A recent NVIDIA GPU, and about 32 GB of video memory 1. Install the Hugging Face hub tools with pip. 2. Download the files with the hf command. 3. Create a separate environment and install vLLM with the card's commands. 4. Serve the model with the command in the next step. ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-NVFP4-Fast ``` ```sh uv venv unsloth-nvfp4-env --python 3.13 source unsloth-nvfp4-env/bin/activate uv pip install "vllm>=0.25.0" "flashinfer-python>=0.6.13" "nvidia-cutlass-dsl>=4.5.2" \ --torch-backend=auto ``` ### How to use it Serve the Fast version with vLLM. The card warns not to use the Marlin backend, since it is about twice as slow. Use the native vLLM backend or the CUTLASS family instead. On a DGX Spark, set the architecture variable and the MoE backend flag, or inference runs about twice as slow. ```text vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast ``` ```sh export CUTE_DSL_ARCH=sm_121a vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast --moe-backend flashinfer_b12x ``` ```sh vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast \ --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' ``` | Tool | What it does | | --- | --- | | vllm serve | Starts a server for the model | | --moe-backend flashinfer\_b12x | Picks the right backend on a DGX Spark | | CUTE\_DSL\_ARCH | Names the chip architecture for the DGX Spark | | --speculative-config | Turns on MTP, so the model drafts its own tokens | | Marlin backend | Avoid it, since it runs about twice as slow | ### Example You create the environment, install vLLM and start the server with the plain command. You send a coding question to the server. The card says the MTP option adds faster decoding, with somewhat lower throughput. ### For newcomers and experts Start with the plain serve command and check that it answers. Add the MTP option only if you care about speed for a single user. The card's throughput numbers come from one B200 GPU with 128 concurrent requests, so your results will differ. The card compares output throughput in tokens per second. Output tokens per second | Label | Value | | --- | --- | | NVIDIA 27B with Marlin | 2403 tokens/s | | Unsloth 27B with cute-DSL | 6863 tokens/s | | NVIDIA 35B-A3B with Marlin | 8721 tokens/s | | Unsloth 35B-A3B with cute-DSL and trtllm | 15636 tokens/s | The card also reports accuracy tests on MMLU-Pro, GPQA and AIME 2025. For this Fast version it lists 85.58 on MMLU-Pro, against 85.75 for BF16. It says the faster builds score about the same as the others. Install vLLM in a separate environment → Serve the NVFP4 Fast model → Send a prompt to the server → Read the reply ### Good to know - Format: Safetensors in NVFP4, made for vLLM and not for GGUF tools. - Speed: The card says 1.79x faster throughput than other NVFP4 builds. - Backend: Do not use Marlin, because it runs about twice as slow. - License: Apache 2.0. - Cost to run: none named. You need a capable NVIDIA GPU, and no paid service appears in the source. - Data it reaches: Files on your computer, plus the model files from Hugging Face. - Not for: Machines without a suitable NVIDIA GPU. ## README ````markdown # Qwen3.6-35B-A3B-NVFP4-Fast Serves Qwen3.6 35B A3B in a fast NVFP4 build by Unsloth for vLLM on NVIDIA GPUs. The card says it has 1.79x the throughput of other NVFP4 builds and fits a 32 GB card. Made by unsloth on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds a fast NVFP4 version of Qwen3.6-35B-A3B, a model that reads text and images and focuses on coding. Unsloth quantized it and calibrated it on its own dataset mixed with UltraChat. The files are in Safetensors format for vLLM, not GGUF. The record lists about 19.8 GB of weights. Who it suits: Teams that serve models on NVIDIA GPUs and care about throughput. The card says it works on a 32 GB graphics card. It also suits developers who want a strong coding model behind a local server. ```table Industry | Who uses it | How they use it | Example Software | Developers | Serve a coding model from their own GPU | Run an agent against a local vLLM server Platform teams | Infrastructure engineers | Raise requests handled per second | Serve many users from one card Research | AI researchers | Compare NVFP4 builds with FP8 and BF16 | Check accuracy against the full precision model Self-hosting | Privacy minded teams | Keep prompts on their own hardware | Run it inside a private network ``` People without an NVIDIA GPU can skip it, because the card's steps target vLLM on NVIDIA hardware. Where to use it: - vLLM on an NVIDIA GPU, as in the card - A DGX Spark machine, with the extra settings the card gives - Windows, macOS or Linux with Python, for downloading the files ```table Use case | Who | What they get back High throughput serving | Platform teams | More output tokens per second from one GPU Agentic coding | Developers | Code changes and repository level reasoning Faster decoding | Developers | Replies drafted with the built in MTP module Image input | Developers | Answers about pictures you send ``` How to set it up: - Python, for the Hugging Face hub tools - A separate Python environment for vLLM, which the card builds with uv - A recent NVIDIA GPU, and about 32 GB of video memory 1. Install the Hugging Face hub tools with pip. 2. Download the files with the hf command. 3. Create a separate environment and install vLLM with the card's commands. 4. Serve the model with the command in the next step. ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-NVFP4-Fast ``` ```sh uv venv unsloth-nvfp4-env --python 3.13 source unsloth-nvfp4-env/bin/activate uv pip install "vllm>=0.25.0" "flashinfer-python>=0.6.13" "nvidia-cutlass-dsl>=4.5.2" \ --torch-backend=auto ``` How to use it: Serve the Fast version with vLLM. The card warns not to use the Marlin backend, since it is about twice as slow. Use the native vLLM backend or the CUTLASS family instead. On a DGX Spark, set the architecture variable and the MoE backend flag, or inference runs about twice as slow. ```text vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast ``` ```sh export CUTE_DSL_ARCH=sm_121a vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast --moe-backend flashinfer_b12x ``` ```sh vllm serve unsloth/Qwen3.6-35B-A3B-NVFP4-Fast \ --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}' ``` ```table Tool | What it does vllm serve | Starts a server for the model --moe-backend flashinfer_b12x | Picks the right backend on a DGX Spark CUTE_DSL_ARCH | Names the chip architecture for the DGX Spark --speculative-config | Turns on MTP, so the model drafts its own tokens Marlin backend | Avoid it, since it runs about twice as slow ``` Example: You create the environment, install vLLM and start the server with the plain command. You send a coding question to the server. The card says the MTP option adds faster decoding, with somewhat lower throughput. For newcomers and experts: Start with the plain serve command and check that it answers. Add the MTP option only if you care about speed for a single user. The card's throughput numbers come from one B200 GPU with 128 concurrent requests, so your results will differ. The card compares output throughput in tokens per second. ```chart Title: Output tokens per second NVIDIA 27B with Marlin: 2403 tokens/s Unsloth 27B with cute-DSL: 6863 tokens/s NVIDIA 35B-A3B with Marlin: 8721 tokens/s Unsloth 35B-A3B with cute-DSL and trtllm: 15636 tokens/s ``` The card also reports accuracy tests on MMLU-Pro, GPQA and AIME 2025. For this Fast version it lists 85.58 on MMLU-Pro, against 85.75 for BF16. It says the faster builds score about the same as the others. ```flow Install vLLM in a separate environment Serve the NVFP4 Fast model Send a prompt to the server Read the reply ``` Good to know: - Format: Safetensors in NVFP4, made for vLLM and not for GGUF tools. - Speed: The card says 1.79x faster throughput than other NVFP4 builds. - Backend: Do not use Marlin, because it runs about twice as slow. - License: Apache 2.0. - Cost to run: none named. You need a capable NVIDIA GPU, and no paid service appears in the source. - Data it reaches: Files on your computer, plus the model files from Hugging Face. - Not for: Machines without a suitable NVIDIA GPU. ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 19.8B - Weights: about 19.8 GB in U8 - Formats: Safetensors - Downloads in the last 30 days: 695,020 - Likes: 134 - License: apache-2.0 - Based on: Qwen/Qwen3.6-35B-A3B - Quantized versions: 4 - Tags: transformers, safetensors, qwen3_5_moe, image-text-to-text, unsloth, qwen, qwen3_5, conversational, 8-bit, compressed-tensors ## Links - Model: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast - Files: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download unsloth/Qwen3.6-35B-A3B-NVFP4-Fast ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [unsloth/Qwen3.6-35B-A3B-NVFP4-Fast on Hugging Face](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.6-35B-A3B-NVFP4-Fast (this page) | 695,020 | 134 | 19.8B | apache-2.0 | 2026-07-12 | | [Qwen3.6-35B-A3B-2Bit-GSQ](https://mindket.com/p/1419ceb9-6ab2-48e2-aec8-e0401259317e) | 974 | 37 | 35.5B | apache-2.0 | 2026-08-14 | | [Qwen3.6-35B-A3B-MTP-GGUF](https://mindket.com/p/90db7fe0-7044-4238-a508-5d2ecb760999) | 2,315,167 | 1,051 | 35.5B | apache-2.0 | 2026-05-20 | | [Qwen3.6-35B-A3B-GGUF](https://mindket.com/p/9ac02ca9-6ba3-4ea0-b3f6-dcaea6e7789f) | 1,649,871 | 1,789 | 34.7B | apache-2.0 | 2026-04-20 | | [Qwopus3.8-27B-Flash-V2-GGUF](https://mindket.com/p/f4542d06-f1ba-46a9-8148-58a34e8911b6) | 897,675 | 75 | 461M | apache-2.0 | 2026-09-21 | | [Qwen3.6-27B-GGUF](https://mindket.com/p/91bfefb7-49da-47e4-a52a-f80091e27806) | 227,770 | 978 | 26.9B | apache-2.0 | 2026-04-22 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Bluesky · Jason Mulligan · 2026-07-11](https://bsky.app/profile/jasonmulligan.com/post/3mqdeqqo57c24): loading huggingface.co/unsloth/Qwen... ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-07-12 - First published: 2026-07-10 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open unsloth/Qwen3.6-35B-A3B-NVFP4-Fast on Hugging Face](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4-Fast) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/dab11469-0f18-43a7-b023-f4a9bfcdea9c) - [Its page on Mindket](https://mindket.com/p/dab11469-0f18-43a7-b023-f4a9bfcdea9c)