MiMo-V2.6-Flash-RL-NVFP4

MiMo-V2.6-Flash-RL-NVFP4

Serves Xiaomi's MiMo-V2.6-Flash-RL as an NVFP4 build for Blackwell GPUs, with the release's exact expert values and more throughput on two 96 GB cards. For people who self-host large models.

Made by primitive-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is an NVFP4 build of XiaomiMiMo's MiMo-V2.6-Flash-RL, a 159B text generation model, packed to 182.40 GB so it runs on two 96 GB cards. Xiaomi already ships the model with 4-bit experts, so this build changes the expert format and keeps every expert value as released. It is meant for Blackwell GPUs.

Who it suits: People who host large models on their own hardware and own two 96 GB Blackwell cards. It fits those who are comfortable with Docker and vLLM.

Industry | Who uses it | How they use it | Example
Self-hosted AI | Infrastructure engineers | Serve a large model on two Blackwell cards | Run an OpenAI-style endpoint for an internal team
AI tooling | Agent developers | Use a model with tool calling | Let an agent call tools through the server's parser
Model evaluation | Researchers | Compare 4-bit formats on one model | Weigh NVFP4 against the release's MXFP4 experts

Anyone without Blackwell GPUs can skip it, because the card says it requires them.

Where to use it:

• A machine with Docker and two NVIDIA Blackwell GPUs with 96 GB each.
• vLLM, through the MiMo image the card names.
• OpenAI-style requests sent to the server on port 8000.
• Windows, macOS or Linux: the file download runs on all three with Python.

Use case | Who | What they get back
Hosted chat | Infrastructure engineers | Replies from a model with a 131,072-token limit as configured
Tool calling | Agent developers | Tool calls parsed by the server's mimo parser
Throughput testing | Researchers | Measured speeds at one and sixteen concurrent requests

How to set it up: What you need first:

• Python with pip, to download the files.
• Docker with GPU access.
• Two 96 GB NVIDIA Blackwell GPUs.
• Room for the files, which total 182.40 GB.

1. Download the model into a local folder with the hf command.
2. Mount the vllm_patch file into the vLLM image, as in the command.
3. Start the container with the model path and the flags shown.

hf download primitive-ai/MiMo-V2.6-Flash-RL-NVFP4 --local-dir ./MiMo-V2.6-Flash-RL-NVFP4

SP=/usr/local/lib/python3.12/dist-packages/vllm
docker run --gpus all --ipc=host --shm-size 32g -p 8000:8000 -v $PWD:/models \
  -v $PWD/MiMo-V2.6-Flash-RL-NVFP4/vllm_patch/mimo_v2.py:$SP/model_executor/models/mimo_v2.py:ro \
  vllm/vllm-openai:mimo-v26-x86_64-cu130 \
  --model /models/MiMo-V2.6-Flash-RL-NVFP4 \
  --tensor-parallel-size 2 --trust-remote-code --generation-config vllm \
  --max-model-len 131072 --gpu-memory-utilization 0.95 \
  --limit-mm-per-prompt '{"image":4,"video":0,"audio":0}' \
  --reasoning-parser mimo --tool-call-parser mimo --enable-auto-tool-choice

The patch file matters. If you leave the mount out, the model fails to load and raises a KeyError. The image's loader expects the release's own FP8 naming.

How to use it: Once the container runs, send chat requests to port 8000. The mimo reasoning and tool-call parsers are switched on, and up to 4 images fit in one prompt. A zero for audio in the limit means the audio encoder is never loaded.

Title: Flags in the serve command
Tool | What it does
--tensor-parallel-size 2 | Splits the model across the two GPUs
--max-model-len 131072 | Sets the longest context the server accepts
--reasoning-parser mimo | Separates the model's thinking from its answer
--tool-call-parser mimo | Turns tool calls into structured output
--limit-mm-per-prompt | Allows 4 images and no video or audio per prompt

Example: You start the container, then send a chat request that includes a tool definition. The server answers through its mimo parsers, which pick out the reasoning and any tool call. The card measured tool-calling accuracy of 78.6, the same as the release.

For newcomers and experts: This is a large model prepared for special hardware, so it is not a casual download. A good first step is to check that you have two 96 GB Blackwell cards before you fetch 182.40 GB.

The release stores experts in MXFP4 with one E8M0 scale per 32 weights. This build swaps that for two E4M3 scales per 16 weights plus a power-of-two global scale. Every value survives without rounding. Each layer gets its own activation scale, because the final MoE layer produces values near 60,000 on the author's prompts. NVFP4 uses 4.5 bits per weight against 4.25, so the build is 9.48 GB larger and the KV cache pool shrinks.

Download the NVFP4 files
Mount the vllm_patch file into the image
Start vLLM on two Blackwell GPUs
Send chat and tool requests to port 8000

Good to know: The author tested on two RTX PRO 6000 Blackwell cards, with 16 concurrent requests for the throughput figure.

Title: Throughput at 16 concurrent requests
Release, MXFP4 experts: 402 tok/s
This build, NVFP4 experts: 532 tok/s

• Scores: knowledge 91.1 against 90.3 for the release, and tool-calling 78.6 for both.
• Cache: the KV pool holds 250,094 tokens, against 627,110 for the release.
• Speculative decoding: the drafter ships as released but was not measured with this build.
• License: mit
• Cost to run: no API key or account named, but it needs two 96 GB Blackwell GPUs.
• Data it reaches: files you download from Hugging Face, and the prompts and images you send to your own server.
• Not for: GPUs older than Blackwell, or anyone who wants a BF16 starting point, since none exists.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: transformers
• Parameters: 159B
• Weights: about 159 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 298
• Likes: 4
• License: mit
• Based on: XiaomiMiMo/MiMo-V2.6-Flash-RL
• Tags: transformers, safetensors, mimo_v2, text-generation, vllm, modelopt, nvfp4, quantized, mimo, mimo-v2.6, agentic, tool-calling

Links

• Model: https://huggingface.co/primitive-ai/MiMo-V2.6-Flash-RL-NVFP4
• Files: https://huggingface.co/primitive-ai/MiMo-V2.6-Flash-RL-NVFP4/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download primitive-ai/MiMo-V2.6-Flash-RL-NVFP4

Then follow the usage in its model card. Nothing in this kit runs by itself.