MOSS-VL-Realtime

MOSS-VL-Realtime

Watches a live video stream and answers questions while frames keep arriving, staying silent when nothing new shows. For developers and researchers working on realtime video understanding.

Made by OpenMOSS-Team on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: MOSS-VL-Realtime is a video text to text model from OpenMOSS-Team that reads a live video stream and answers questions while frames keep arriving. It is the streaming version in the MOSS-VL line, a family that also handles images and long videos. It has 11.3B parameters, handles English and Chinese, and is built on MOSS-VL-Base.

Who it suits: It suits people who build tools that watch a video feed and need answers as events unfold, and researchers who study streaming video understanding.

Industry | Who uses it | How they use it | Example
Research | Vision-language researchers | Study streaming video understanding and answers that change as a scene develops | Asking a question mid stream and checking whether the model revises its earlier reply
Software | Application developers | Prototype a tool that takes video frames as they come and answers questions | A test app that handles a question asked at any timestamp
Education | Lab instructors | Show students how a model reasons about event order and duration | A class demo that asks how long an event in a clip lasted

You can skip it if you only work with still images or finished videos, since the same family has offline models for that.

Where to use it:

• Python with the transformers library, with trust_remote_code turned on
• A conda environment made for the MOSS-VL repository
• Windows, macOS or Linux with Python, to download the files
• Your own app, which feeds it frames as PIL-compatible images with timestamps

Use case | Who | What they get back
Questions on a live stream | Developers | Answers based on the frames seen so far
Watching without chatter | App builders | No reply when nothing new shows, and the model keeps observing
Timing questions | Researchers | Answers about event order, duration and pacing

How to set it up:

• Python 3.12 inside a conda environment
• Memory for the weights: about 22.7 GB in BF16, or about 5.7 GB at 4-bit by estimate
• The MOSS-VL project repository from OpenMOSS and its requirements file

1. Download the model files with the Hugging Face hub package.
2. Clone the MOSS-VL repository from OpenMOSS and move into its folder.
3. Create and activate a conda environment with Python 3.12.
4. Install the requirements.txt file from that repository with pip.
5. Load the processor and model with transformers.

pip install -U huggingface_hub
hf download OpenMOSS-Team/MOSS-VL-Realtime

The environment commands from the model card:

conda create -n moss_vl python=3.12 pip -y
conda activate moss_vl

How to use it: Load the processor with the checkpoint name, then feed the model timestamped frames one by one and ask your question whenever you like. The card describes an inference interface for this and allows one active realtime session per model instance. Its later sections hold the full streaming calls.

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

checkpoint = "OpenMOSS-Team/MOSS-VL-Realtime"

processor = AutoProcessor.from_pretrained(
    checkpoint,
    trust_remote_code=True,
    frame_extract_num_threads=1,
)

Tool | What it does
hf download | Fetches the model files to your computer
AutoProcessor | Prepares frames and text for the model
AutoModelForCausalLM | Loads the model weights

Example: While a video stream runs, you ask what the person on screen is holding right now. The model answers from the frames it has seen so far. If nothing new has appeared, it can stay silent and keep watching, and it may correct an earlier answer when later frames change the picture.

For newcomers and experts: It is a model that watches a video as it plays instead of waiting for the whole file. For a first try, download the files with the two commands above, follow the model card to load the processor and model, then give it a few timestamped frames and one question. Plan for a GPU with plenty of memory, or a 4-bit copy.

Underneath, a cross-attention design keeps visual encoding apart from language reasoning, so new frames can join the context while the model is already writing. Each frame carries an absolute timestamp. A position scheme called XRoPE places text tokens and visual patches in one space of time, height and width. The card lists a 256K context, 1.0 frames per second and up to 256 frames by default, a vision patch size of 16 and a temporal patch size of 1.

Frames arrive with timestamps
The model reads frames while it writes
You ask a question at any moment
It answers, stays silent or revises

Good to know: The model needs a large GPU, and the quantized copies shrink the weights a lot.

Title: Weight size in GB
BF16: 22.7 GB
4-bit, estimate: 5.7 GB

• License: Apache 2.0
• Cost to run: no API key or paid service is named, you supply the hardware
• Data it reaches: downloads weights from Hugging Face, runs code shipped in the model repository because of trust_remote_code, and reads the video frames you give it
• Not for: running several realtime sessions on one model instance, since the card allows one at a time

On Hugging Face, 13 November 2026

• Task: video text to text
• Library: transformers
• Parameters: 11.3B
• Weights: about 22.7 GB in BF16; about 5.7 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,934
• Likes: 117
• License: apache-2.0
• Languages: en, zh
• Based on: OpenMOSS-Team/MOSS-VL-Base
• Quantized versions: 2
• Fine-tunes: 1
• Tags: transformers, safetensors, moss_vl, feature-extraction, realtime, streaming, video-understanding, image-understanding, moss-vl, openmoss, multimodal, video

Links

• Model: https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime
• Files: https://huggingface.co/OpenMOSS-Team/MOSS-VL-Realtime/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download OpenMOSS-Team/MOSS-VL-Realtime

Then follow the usage in its model card. Nothing in this kit runs by itself.