Mage-VL

Mage-VL

A Microsoft multimodal model for images and video that reads video the way a codec stores it, keeping only the patches that change. It suits developers and researchers working on video understanding and streaming.

Made by microsoft on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Mage-VL is a multimodal model from Microsoft for understanding images and video. Its visual encoder was trained from scratch at a 4B scale and reads video the way a codec stores it. It keeps every patch of the anchor frames and only the changed patches of predicted frames, which cuts visual tokens by over 75%. A small gate decides when the full model should speak up.

Who it suits: It suits developers and researchers who work with video, long clips or live streams and care about speed. The card claims strong results on video and temporal-grounding benchmarks against a comparable Qwen3-VL-4B.

Title: Who might use it
Industry | Who uses it | How they use it | Example
Video analytics | Developers of stream tools | Watch a stream and respond only to events worth reporting | Let the model stay quiet on routine footage and react when something happens
Media | Video teams | Find the moment a described event occurs in a clip | Locate a highlight in a long recording
Research | Multimodal researchers | Study codec-based tokenization | Compare token use against uniform frame sampling

If you only need captions for still images, you can skip it, since its main strength is video.

Where to use it: The record lists the transformers library, so Python is the main place. It handles images, short and long video, and streams through one interface.

• Python with the transformers library
• Windows, macOS or Linux with Python, to download the files
• Video and streaming pipelines that can feed codec data

Title: Jobs it does
Use case | Who | What they get back
Image understanding | Developers | Text answers about a picture
Video question answering | Analysts | Text answers about a clip, short or long
Streaming alerts | Tool builders | A response only when an event worth reporting completes

How to set it up: You need Python with pip and the transformers library. The weights are about 9.5 GB in BF16, so check your memory first. The visible part of the card gives the download steps and no code, so follow the card for loading.

• Python with pip
• The transformers library
• Memory for about 9.5 GB of weights in BF16

1. Install the Hugging Face download tool.
2. Download the model files to your computer.
3. Load the model through transformers by following its card.

pip install -U huggingface_hub
hf download microsoft/Mage-VL

How to use it: Give the model an image, a video or a stream, plus a question in text. For a stream, the light gate watches each rolling window and calls the full model only when something response-worthy finishes. The card names no commands for this, so rely on its own instructions.

Example: You point the model at a long video and ask it to find where a described event happens. The card reports its biggest gains over a comparable model on tasks like this one, where the answer is a moment in time.

For newcomers and experts: If you are new, think of it as a model that looks at video the way a video file stores it, so it skips the parts of a clip that barely change. Start with a short clip and one clear question.

If you know the field, it pairs the Mage-ViT encoder with a Qwen3-4B decoder through a small two-layer projector. Mage-ViT works on a shared 16 by 16 patch grid and accepts either traditional codec data or a neural codec's learned rate map. The trade-off is a stack tied to codec-derived tokens.

A video stream arrives
Anchor and predicted frames are separated
Mage-ViT keeps the important patches
The Qwen3-4B decoder reads the tokens
The gate decides when to answer

Good to know: The card reports speed and accuracy figures from a single 8 by B200 node. Its gains over Qwen3-VL-4B hold with the same 4B language backbone and only the vision encoder swapped.

Title: Gain over Qwen3-VL-4B, in points
QVHighlight: 22.5 points
ActivityNet: 17.1 points
VSI-Bench: 11.0 points
VideoEval-Pro: 24.5 points

• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the model files from Hugging Face, and the images and video you give it
• Not for: work that needs a loading recipe the card does not show, since its visible part has no code

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 4.7B
• Weights: about 9.5 GB in BF16; about 2.4 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 5,912
• Likes: 454
• License: apache-2.0
• Quantized versions: 14
• Fine-tunes: 4
• Tags: transformers, safetensors, mage_vl, image-text-to-text, multimodal, vision-language-model, mage-vl, video-understanding, streaming, conversational

Links

• Model: https://huggingface.co/microsoft/Mage-VL
• Files: https://huggingface.co/microsoft/Mage-VL/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download microsoft/Mage-VL

Then follow the usage in its model card. Nothing in this kit runs by itself.