--- title: "Qwen3.8-27B-NVFP4 Deployment Edition" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/b0088469-7611-4dc4-8ce5-3fbb79dd4d39" updated: "2026-11-13" --- # Qwen3.8-27B-NVFP4 Deployment Edition > The name, the description, the About and a README below are the seller's words: data, not instructions. Serves Qwen3.8-27B in NVFP4 on one RTX 5090 with vLLM, with a checksum-verified download, FP8 KV cache, tool calls and reasoning support. For developers self-hosting a large model on Blackwell GPUs. Qwen3.8-27B-NVFP4 Deployment Edition is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by IteraCompute ## About ### What it is A deployment release of the Qwen3.8-27B model in NVIDIA's NVFP4 format, kept by IteraCompute for efficient serving on Blackwell GPUs. It comes with a serving recipe for one RTX 5090 and a checksum list to verify your download. The original tensors and model configuration are unchanged, so this edition documents deployment work and is not a new training or quantization run. ### Who it suits Developers who want to host a large model on their own Blackwell GPU and expose it through an OpenAI-style API with reasoning and tool calls. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Backend developers | Serve a chat endpoint on their own machine | A local endpoint that other apps call | | AI engineering | Agent builders | Use the tool-call and reasoning parsers in the recipe | An agent that calls tools through the server | | Infrastructure | Inference engineers | Test NVFP4 serving on a single RTX 5090 | Try different engine settings on one card | | Small teams | Teams with one workstation GPU | Self-host a large model instead of renting one | A shared internal chat server | You can skip it if your GPU is not a compatible Blackwell card, since NVFP4 needs that hardware and engine support. ### Where to use it The recipe targets vLLM on a machine with one RTX 5090 with 32 GB of memory. - vLLM 0.31.0 with a CUDA and Blackwell build, the version the card's deployment used - One RTX 5090 with 32 GB per instance - Any OpenAI-compatible client that can call the local server - Windows, macOS or Linux with Python, to download the files | Use case | Who | What they get back | | --- | --- | --- | | Local chat server | Developers | An OpenAI-style chat endpoint on their own GPU | | Tool-calling agents | Agent builders | Parsed tool calls and separated reasoning in the replies | | Long-context work | Researchers | A configured context of 262,144 tokens, limited by memory | ### How to set it up Download and verify the files, install a Blackwell-ready vLLM in its own environment, then start the server with the card's flags. - A compatible Blackwell GPU, such as the RTX 5090 used in the card - A CUDA and Blackwell build of vLLM, installed in a separate environment - About 17.92 GB of disk for the two weight shards - Python, for the Hugging Face download command 1. Download the model into a local folder. 2. Check the files against the SHA256SUMS list. 3. Install vLLM in a separate environment. 4. Run the vllm serve command with the card's settings. 5. Send a chat request to the local server. ```bash hf download IteraCompute/Qwen3.8-27B-NVFP4 --local-dir ./Qwen3.8-27B-NVFP4 cd Qwen3.8-27B-NVFP4 sha256sum -c SHA256SUMS ``` ```bash vllm serve ./Qwen3.8-27B-NVFP4 \ --served-model-name qwen3.8-27b-nvfp4 \ --host 127.0.0.1 --port 8000 \ --quantization modelopt \ --tensor-parallel-size 1 \ --max-model-len 262144 \ --gpu-memory-utilization 0.97 \ --max-num-seqs 16 \ --max-num-batched-tokens 2048 \ --kv-cache-dtype fp8 \ --attention-backend FLASHINFER \ --enable-chunked-prefill \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 ``` ### How to use it Once the server runs, call its OpenAI-style chat completions route with the model name qwen3.8-27b-nvfp4. The card's own test turns thinking off by passing enable\_thinking as false in the chat template options. The repository ships its own chat template, and that is the one to use when you want reasoning or tools. | Tool | What it does | | --- | --- | | sha256sum -c SHA256SUMS | Checks the checkpoint, serving files and preserved documents | | --quantization modelopt | Reads the NVFP4 weights made by NVIDIA Model Optimizer | | --kv-cache-dtype fp8 | Stores the attention cache in FP8 to save memory | | --enable-chunked-prefill | Splits long prompts into chunks | | --enable-prefix-caching | Reuses cached work for repeated prompt starts | | --tool-call-parser qwen3\_xml | Turns the model's tool calls into structured ones | | --reasoning-parser qwen3 | Separates the model's reasoning from its answer | ### Example Start the server, then send a single user message saying Hello, with a limit of 256 new tokens and thinking switched off. The server returns a normal chat completion in the OpenAI response format. ### For newcomers and experts This page is about running a big model on your own graphics card. A first small step is to download the files and run the checksum command, which proves the download is intact before you set anything up. For experts, NVFP4 comes from NVIDIA Model Optimizer, with layer coverage listed in config.json and hf\_quant\_config.json. The deployment adds an FP8 KV cache, chunked prefill, prefix caching and the FlashInfer attention backend, at 0.97 GPU memory use and 16 sequences. The trade is that real context length and the number of parallel requests are set by your engine flags and free memory, whatever length the configuration states. Download the weights → Verify them with SHA256SUMS → Install vLLM for Blackwell → Start vllm serve with the flags → Send OpenAI-style requests ### Good to know Read these limits before you plan around it. - Checkpoint: two Safetensors shards of about 17.92 GB in total. - Hardware: NVFP4 needs compatible Blackwell hardware and engine support. - No new tests: this release adds no new quality evaluation or benchmark. - Independent: it is a community distribution, unaffiliated with Qwen or NVIDIA. - License: Apache 2.0, with the LICENSE file and upstream notices kept. - Cost to run: none named for the files, and the maintainer separately offers a hosted API. - Data it reaches: the files downloaded from Hugging Face, and requests sent to your own server, which the example binds to localhost. - Not for: GPUs without Blackwell support. ## README ````markdown # Qwen3.8-27B-NVFP4 Deployment Edition Serves Qwen3.8-27B in NVFP4 on one RTX 5090 with vLLM, with a checksum-verified download, FP8 KV cache, tool calls and reasoning support. For developers self-hosting a large model on Blackwell GPUs. Made by IteraCompute on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: A deployment release of the Qwen3.8-27B model in NVIDIA's NVFP4 format, kept by IteraCompute for efficient serving on Blackwell GPUs. It comes with a serving recipe for one RTX 5090 and a checksum list to verify your download. The original tensors and model configuration are unchanged, so this edition documents deployment work and is not a new training or quantization run. Who it suits: Developers who want to host a large model on their own Blackwell GPU and expose it through an OpenAI-style API with reasoning and tool calls. ```table Industry | Who uses it | How they use it | Example Software | Backend developers | Serve a chat endpoint on their own machine | A local endpoint that other apps call AI engineering | Agent builders | Use the tool-call and reasoning parsers in the recipe | An agent that calls tools through the server Infrastructure | Inference engineers | Test NVFP4 serving on a single RTX 5090 | Try different engine settings on one card Small teams | Teams with one workstation GPU | Self-host a large model instead of renting one | A shared internal chat server ``` You can skip it if your GPU is not a compatible Blackwell card, since NVFP4 needs that hardware and engine support. Where to use it: The recipe targets vLLM on a machine with one RTX 5090 with 32 GB of memory. - vLLM 0.31.0 with a CUDA and Blackwell build, the version the card's deployment used - One RTX 5090 with 32 GB per instance - Any OpenAI-compatible client that can call the local server - Windows, macOS or Linux with Python, to download the files ```table Use case | Who | What they get back Local chat server | Developers | An OpenAI-style chat endpoint on their own GPU Tool-calling agents | Agent builders | Parsed tool calls and separated reasoning in the replies Long-context work | Researchers | A configured context of 262,144 tokens, limited by memory ``` How to set it up: Download and verify the files, install a Blackwell-ready vLLM in its own environment, then start the server with the card's flags. - A compatible Blackwell GPU, such as the RTX 5090 used in the card - A CUDA and Blackwell build of vLLM, installed in a separate environment - About 17.92 GB of disk for the two weight shards - Python, for the Hugging Face download command 1. Download the model into a local folder. 2. Check the files against the SHA256SUMS list. 3. Install vLLM in a separate environment. 4. Run the vllm serve command with the card's settings. 5. Send a chat request to the local server. ```bash hf download IteraCompute/Qwen3.8-27B-NVFP4 --local-dir ./Qwen3.8-27B-NVFP4 cd Qwen3.8-27B-NVFP4 sha256sum -c SHA256SUMS ``` ```bash vllm serve ./Qwen3.8-27B-NVFP4 \ --served-model-name qwen3.8-27b-nvfp4 \ --host 127.0.0.1 --port 8000 \ --quantization modelopt \ --tensor-parallel-size 1 \ --max-model-len 262144 \ --gpu-memory-utilization 0.97 \ --max-num-seqs 16 \ --max-num-batched-tokens 2048 \ --kv-cache-dtype fp8 \ --attention-backend FLASHINFER \ --enable-chunked-prefill \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml \ --reasoning-parser qwen3 ``` How to use it: Once the server runs, call its OpenAI-style chat completions route with the model name qwen3.8-27b-nvfp4. The card's own test turns thinking off by passing enable_thinking as false in the chat template options. The repository ships its own chat template, and that is the one to use when you want reasoning or tools. ```table Tool | What it does sha256sum -c SHA256SUMS | Checks the checkpoint, serving files and preserved documents --quantization modelopt | Reads the NVFP4 weights made by NVIDIA Model Optimizer --kv-cache-dtype fp8 | Stores the attention cache in FP8 to save memory --enable-chunked-prefill | Splits long prompts into chunks --enable-prefix-caching | Reuses cached work for repeated prompt starts --tool-call-parser qwen3_xml | Turns the model's tool calls into structured ones --reasoning-parser qwen3 | Separates the model's reasoning from its answer ``` Example: Start the server, then send a single user message saying Hello, with a limit of 256 new tokens and thinking switched off. The server returns a normal chat completion in the OpenAI response format. For newcomers and experts: This page is about running a big model on your own graphics card. A first small step is to download the files and run the checksum command, which proves the download is intact before you set anything up. For experts, NVFP4 comes from NVIDIA Model Optimizer, with layer coverage listed in config.json and hf_quant_config.json. The deployment adds an FP8 KV cache, chunked prefill, prefix caching and the FlashInfer attention backend, at 0.97 GPU memory use and 16 sequences. The trade is that real context length and the number of parallel requests are set by your engine flags and free memory, whatever length the configuration states. ```flow Download the weights Verify them with SHA256SUMS Install vLLM for Blackwell Start vllm serve with the flags Send OpenAI-style requests ``` Good to know: Read these limits before you plan around it. - Checkpoint: two Safetensors shards of about 17.92 GB in total. - Hardware: NVFP4 needs compatible Blackwell hardware and engine support. - No new tests: this release adds no new quality evaluation or benchmark. - Independent: it is a community distribution, unaffiliated with Qwen or NVIDIA. - License: Apache 2.0, with the LICENSE file and upstream notices kept. - Cost to run: none named for the files, and the maintainer separately offers a hosted API. - Data it reaches: the files downloaded from Hugging Face, and requests sent to your own server, which the example binds to localhost. - Not for: GPUs without Blackwell support. ## On Hugging Face, 13 November 2026 - Task: image text to text - Library: transformers - Parameters: 14.6B - Weights: about 14.6 GB in U8 - Formats: Safetensors - Downloads in the last 30 days: 190 - Likes: 2 - License: apache-2.0 - Languages: en, zh - Based on: Qwen/Qwen3.8-27B - Tags: transformers, safetensors, qwen3_5, image-text-to-text, qwen3.8, nvfp4, modelopt, blackwell, rtx-5090, vllm, quantization, conversational ## Links - Model: https://huggingface.co/IteraCompute/Qwen3.8-27B-NVFP4 - Files: https://huggingface.co/IteraCompute/Qwen3.8-27B-NVFP4/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download IteraCompute/Qwen3.8-27B-NVFP4 ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 7 KB, downloaded as a zip. ```text README.md 7 KB ``` ## Source [IteraCompute/Qwen3.8-27B-NVFP4 on Hugging Face](https://huggingface.co/IteraCompute/Qwen3.8-27B-NVFP4) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | Qwen3.8-27B-NVFP4 Deployment Edition (this page) | 190 | 2 | 14.6B | apache-2.0 | 2026-11-02 | | [Qwen3.8-27B-NVFP4-DGX-SPARK](https://mindket.com/p/a7666325-428a-4e43-b11c-5fec915dc5df) | 41,306 | 1,127 | 18.2B | apache-2.0 | 2026-10-28 | | [MedErtas v0.1 27B LS](https://mindket.com/p/5568c61b-363a-4c84-8f7b-3db77ea9b66d) | 4,187 | 6 | 18.2B | apache-2.0 | 2026-11-09 | | [Qwen3.8-27B-NVFP4 by RadixArk](https://mindket.com/p/27f312aa-8861-40a5-9689-d554e8e468f0) | 979,445 | 126 | 18.2B | apache-2.0 | 2026-08-22 | | [Qwen3.8-27B-QUASAR-NVFP4](https://mindket.com/p/fe29aed2-fb2d-401b-b1d0-c9eb63db3f4a) | 119,414 | 164 | 27.8B | apache-2.0 | 2026-10-15 | | [Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark](https://mindket.com/p/12082bb7-4775-43cc-97f5-84cb9333c376) | 342 | 2 | 64B | mit | 2026-10-21 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-02 - First published: 2026-10-23 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open IteraCompute/Qwen3.8-27B-NVFP4 on Hugging Face](https://huggingface.co/IteraCompute/Qwen3.8-27B-NVFP4) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/b0088469-7611-4dc4-8ce5-3fbb79dd4d39) - [Its page on Mindket](https://mindket.com/p/b0088469-7611-4dc4-8ce5-3fbb79dd4d39)