Realtime-Venus

Realtime-Venus

Holds two 9B models for live conversation that listen and speak at the same time, handle interruptions and hand tasks to outside tools. One watches video and audio, the other takes audio only.

Made by inclusionAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Realtime-Venus is a system for live conversation in which the model keeps listening while it speaks and can hand tasks to outside tools without pausing the talk. This repository holds two 9B checkpoints. Realtime-Venus-Omni watches and listens, and Realtime-Venus-Audio works from audio and text. Both can answer in text or speech.

Who it suits: It suits developers and researchers who build voice or camera assistants that must cope with interruptions and act on their own timing.

Industry | Who uses it | How they use it | Example
Voice assistants | Teams building spoken helpers | Use the Audio checkpoint for conversation | Let a user cut in mid-reply and change the request
Video assistants | Teams building helpers that watch a stream | Use the Omni checkpoint to watch and listen | Have it speak up when something in the video calls for it
Dialogue research | Researchers of full-duplex conversation | Study delegation on one shared timeline | Test how backend results arrive without blocking the talk

If you only need a plain text chat model, you can skip it.

Where to use it:

• Python projects that use the transformers library with the custom code in each folder
• Audio assistants that need text or speech replies
• Camera or video assistants that need audio and visual input
• The GitHub repository and ModelScope page that the card links to
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Spoken conversation | Voice app teams | Replies that handle interruptions, corrections and backchannels
Proactive watching | Video app teams | A model that decides when to respond without a prompt
Long video recall | Researchers | Evidence pulled from earlier moments of a long video

How to set it up: What you need first:

• Python and the Hugging Face command line
• A GPU able to hold a 9B model in BF16
• The runtime from the project's GitHub repository, if you want delegated tasks to run

1. Install the Hugging Face tools with pip.
2. Download the repository, which holds both checkpoint folders.
3. Pick the Omni folder for video, audio and text, or the Audio folder for audio and text.
4. Follow the model card and the GitHub repository for the usage code.

pip install -U huggingface_hub
hf download inclusionAI/Realtime-Venus

How to use it: The visible card text gives no run commands, so use the folder that fits your input and follow its usage notes. The model keeps perceiving while it talks, and it can send in-stream delegation requests that a separate runtime carries out.

Tool | What it does
Realtime-Venus-Omni | The 9B audio-visual model that watches, listens and decides when to speak
Realtime-Venus-Audio | The audio-focused checkpoint for audio understanding and conversation
Token2wav resources | Turn the generated speech tokens into audio with a reference voice

Example: You talk to the Audio checkpoint and interrupt it halfway through a reply to correct yourself. The card says it tells interruptions and corrections from simple backchannels, so it can take your correction and carry on.

For newcomers and experts: A good first try is the Audio checkpoint, because it needs only sound and text. Try speaking over its reply to see how it reacts.

For experts, Omni is adapted from MiniCPM-o 4.5, with a SigLIP2 visual encoder, a Whisper-Medium audio encoder and a Qwen3-8B language backbone. Speech comes from discrete S3 speech tokens through a streaming flow-matching decoder. Delegation requests and their results share one causal timeline with the conversation. The catch is that executing those requests needs the separate runtime.

Audio and video stream in
The model decides whether and when to respond
Text and speech go out on one timeline
Delegation requests go to the runtime
Results return in the same stream

Good to know:

Item | Omni | Audio
Parameters | 9B | 9B
Inputs | Video, images, audio and text | Audio and text
Outputs | Text and optional speech | Text and speech
Context length | 40,960 tokens | 40,960 tokens
Weight type | BF16 | BF16

• Memory: long-video memory needs no extra training, and it archives informative moments to rebuild context later.
• Languages: English and Chinese are listed.
• Results: the card sends you to the technical report for its evaluation numbers.
• License: apache-2.0
• Cost to run: no API key or paid service named.
• Data it reaches: it downloads files from Hugging Face, takes the audio and video you feed it, and reaches outside tools only through the separate runtime.
• Not for: running delegated tasks without that runtime.

On Hugging Face, 13 November 2026

• Task: any to any
• Library: transformers
• Formats: Safetensors
• Downloads in the last 30 days: 2,101
• Likes: 135
• License: apache-2.0
• Languages: en, zh
• Quantized versions: 1
• Fine-tunes: 1
• Tags: transformers, onnx, safetensors, multimodal, audio, video, speech, streaming, full-duplex, long-video, custom-code, any-to-any

Links

• Model: https://huggingface.co/inclusionAI/Realtime-Venus
• Files: https://huggingface.co/inclusionAI/Realtime-Venus/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download inclusionAI/Realtime-Venus

Then follow the usage in its model card. Nothing in this kit runs by itself.