Kandinsky S2V 1.0 Lite

Kandinsky S2V 1.0 Lite

Turns one reference picture and a speech recording into a talking avatar video, animating the whole figure to match the audio. For researchers and developers with a strong GPU. Use only faces and voices you have the right to use.

Made by kandinskylab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Kandinsky S2V 1.0 Lite makes avatar videos from two inputs, a reference frame and a speech track. It animates the whole figure in the picture, not only the head, so the person moves in step with the speech. The audio only guides the motion, and the model does not create any sound itself.

Who it suits: It suits researchers and developers who build talking-avatar tools and have a high-end GPU to run a video model.

Industry | Who uses it | How they use it | Example
Video production | Creators | Animate a character from one picture and a voice track | Making a presenter clip from a still image
Education | Course builders | Turn a narrated script into a talking figure | Putting a face on a recorded lesson
Game and media | Developers | Test speech-driven animation of characters | Animating a cartoon figure to a dialogue line
AI research | Researchers | Study speech-to-video diffusion models | Comparing the standard and distilled versions

You can skip it if you need sound generated, if you lack a large GPU, or if you do not have consent to animate the person in your picture.

Where to use it:

• A GPU machine with the kandinsky-s2v code from GitHub
• The Hugging Face demo space the card lists, for a quick look
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Talking avatar | Creators | A 25 fps video that matches your speech track
Long narration | Producers | A longer video built in 5-second chunks
Faster generation | Engineers | Lower time per second of video by splitting across GPUs

How to set it up: You work from the project's GitHub code and the uv tool. The first run fetches the checkpoint and the helper models by itself.

• A strong NVIDIA GPU, since the card measured about 57 GiB of peak memory on an H100
• The uv tool and Python
• The kandinsky-s2v repository from GitHub
• Room for the 4.2 GiB checkpoint plus the text encoders and the VAE

1. Clone the kandinsky-s2v repository and open its folder.
2. Install the dependencies with uv.
3. Run the generate command with a speech file and a reference image.

cd kandinsky-s2v
uv sync

pip install -U huggingface_hub
hf download kandinskylab/Kandinsky-S2V-1.0-Lite

How to use it: You run one command with a config file, a speech track, a reference frame and an output path. On its first run the code fetches this checkpoint, the text encoders and the VAE from their Hugging Face repositories.

uv run kandy-s2v generate \
    --config src/kandinsky_s2v/configs/s2v_lite.yaml \
    --audio speech.wav \
    --image reference.png \
    --out out.mp4

Tool | What it does
uv sync | Installs the project's dependencies
kandy-s2v generate | Makes a video from a speech track and a reference image
--config | Chooses the settings file for the Lite model
--audio | The speech track that drives the motion
--image | The reference frame to animate
--out | Where the finished video is saved

Example: You pass a short speech.wav and a clear reference.png of a person you have consent to animate. The command writes out.mp4, a talking video of that person at 25 fps, moving in time with the speech.

For newcomers and experts: If you are new to this, you give the model a photo and a recording, and it makes the person in the photo appear to say the words. A first small try is one clear photo and a few seconds of speech.

For people who know the field, it is latent diffusion with flow matching. Qwen2.5-Omni-7B supplies the text and audio conditions, CLIP adds a pooled text embedding, and a HunyuanVideo 3D VAE encodes and decodes. A 2.3B DiT is the backbone, with speech cross-attention adapters on 13 of its 32 visual blocks.

Title: Time to generate one second of video on one H100
Default: 44 s
With compile: 35 s
Split across 4 GPUs: 14 s

Reference frame and speech track go in
Qwen2.5-Omni and CLIP supply the conditions
The diffusion transformer builds video latents
The VAE decodes them into a 25 fps video

Good to know: A distilled sibling called Kandinsky-S2V-1.0-Lite-distill needs 16 steps instead of 50. Memory peaks mostly in the VAE decode, and tiling it lowers the peak at the cost of faint seams.

• License: MIT
• Cost to run: none named, though one H100 used about 57 GiB of memory in the card's tests
• Data it reaches: your reference image and speech file, plus models the code downloads from the Hub on first run
• Not for: animating anyone without their consent, or passing the result off as a genuine recording

On Hugging Face, 13 November 2026

• Task: image to video
• Downloads in the last 30 days: 73
• Likes: 9
• License: mit
• Tags: kandinsky_s2v, speech-to-video, avatar, video-generation, kandinsky, image-to-video

Links

• Model: https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite
• Files: https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download kandinskylab/Kandinsky-S2V-1.0-Lite

Then follow the usage in its model card. Nothing in this kit runs by itself.