Qwen3.8-27B-NVFP4-DGX-SPARK

Qwen3.8-27B-NVFP4-DGX-SPARK

Speeds up Qwen3.8-27B NVFP4 decoding about 3.9 times on a DGX Spark, using the same weights and a tuned speculative-decoding setup. For teams serving it with vLLM on Blackwell GPUs.

Made by autotrust on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository ships the same weights as NVIDIA's Qwen3.8-27B NVFP4 checkpoint, plus a tuned setup for speculative decoding with the model's multi-token prediction head. On a DGX Spark, single-stream decoding rises from 12.5 to 48.2 tokens per second. The card says output tokens are identical at temperature 0, because the layers and the head used for checking are never touched.

Who it suits: Teams that serve this 27B model with vLLM on Blackwell hardware and care about how fast a single stream of text arrives.

Industry | Who uses it | How they use it | Example
Software | Inference engineers | Serve one stream of text faster on a DGX Spark | A coding assistant that streams reasoning and code
AI infrastructure | MLOps teams | Tune the number of speculative tokens for their workload | A tuned k value for chat or for code
Research | Systems researchers | Study the cost of drafting on bandwidth-bound machines | Read the card's byte ledger for the drafter
Developers | Agent builders | Run a local reasoning model with lower latency | A test-running code agent

You can skip it if you do not have Blackwell GPUs, since the card verified the setup on that generation only.

Where to use it: It runs in vLLM, version 0.31.0 or later, on NVIDIA Blackwell machines such as the DGX Spark and the B200.

• vLLM 0.31.0 or later
• A DGX Spark, or an NVIDIA B200 for much higher throughput
• Other Blackwell parts, which the card verified for SM 12.0 and 12.1

Use case | Who | What they get back
Faster single-stream decoding | Developers | About 3.9 times the baseline speed on a DGX Spark
Code and math generation | Agent builders | The card's best speedups, at 4 to 5 times
Open-ended chat | Product teams | A smaller gain, at 3 to 4 times

How to set it up: You need a Blackwell GPU machine with vLLM, and about 21.9 GB for the weights.

• A Blackwell GPU machine
• vLLM 0.31.0 or later
• About 21.9 GB of disk space for the weights

1. Install the Hugging Face download tool.
2. Download the repository, which holds the weights, the draft-vocabulary mask and the patch scripts.
3. Follow the quick start in the card to apply the drafter patch and start vLLM with speculative decoding.

pip install -U huggingface_hub
hf download autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK

How to use it: Serve the model with vLLM and speculative decoding turned on, then send ordinary requests. The right number of speculative tokens depends on the machine and the temperature, so you should benchmark it on your own workload.

Tool | What it does
MTP speculative decoding | Drafts several tokens ahead and checks them in one pass
Draft vocabulary mask | A 64,064 row list that shrinks what the drafter reads
Drafter patch scripts | Apply the cheaper drafter setup
BENCH_REPORT.md | Holds the byte ledger and acceptance curves

Example: On the B200, the card shows the model streaming its thinking and code for a Dijkstra shortest paths task. The answer was 396 tokens in 0.88 seconds, about 497 tokens per second, and the generated program passed its tests.

For newcomers and experts: Speculative decoding lets a small helper guess several words ahead while the big model checks the guesses in one go. When the guesses are right, text comes out faster, and the final text stays the same. A first small try is to follow the quick start and compare tokens per second with and without it.

For experts, native MTP stops improving near 37 tokens per second because the drafter reads the full 248k vocabulary head on every draft step. This recipe cuts the bytes read per step from about 3.4 GB to about 0.42 GB with a 64k draft vocabulary, an NVFP4 draft head and NVFP4 MTP layers. The verification head is untouched, so the output is lossless. The trade is that acceptance drops at higher sampling temperatures.

The cheap drafter proposes several tokens
The full model checks them in one pass
Accepted tokens are kept as output
The first wrong guess is replaced
Output matches decoding without drafting

Good to know: The speed numbers are single-stream results at temperature 0 on a DGX Spark, from the card's own benchmark.

• Temperature: At temperature 0.7 the card advises 4 to 6 speculative tokens and a fresh benchmark.
• B200: The same recipe adds about 4 percent there, against about 28 percent on the DGX Spark.
• Weights: They match the upstream checkpoint exactly, mixed NVFP4 and FP8.

Title: Single-stream tokens per second on a DGX Spark
No MTP: 12.5 tok/s
Native MTP with k=7: 37.4 tok/s
Native MTP with k=6: 37.5 tok/s
This recipe with k=8: 48.2 tok/s

Title: Single-stream tokens per second on a B200
No MTP: 128.6 tok/s
Native MTP with k=7: 444.0 tok/s
This recipe with k=7: 463.5 tok/s

• License: Apache 2.0.
• Cost to run: none named, the source lists no API key, account or paid service, though it needs a Blackwell GPU.
• Data it reaches: only the prompts you send to your own vLLM server, plus the files you download.
• Not for: expecting the full speedup at high sampling temperatures, since acceptance drops, or running on GPUs the card did not verify.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: vllm
• Parameters: 18.2B
• Weights: about 18.2 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 41,306
• Likes: 1,127
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: vllm, safetensors, qwen3_5, qwen, qwen3.8, nvfp4, fp4, speculative-decoding, modelopt, dgx-spark, gb10, blackwell

Links

• Model: https://huggingface.co/autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK
• Files: https://huggingface.co/autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download autotrust/Qwen3.8-27B-NVFP4-DGX-SPARK

Then follow the usage in its model card. Nothing in this kit runs by itself.