Kandinsky S2V 1.0 Lite distilled

Kandinsky S2V 1.0 Lite distilled

Turns one reference image and a speech recording into an avatar video, animating the whole figure to match the speech. The distilled Lite version runs 16 steps and is about 5 times faster than the base.

Made by kandinskylab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Kandinsky S2V 1.0 Lite distilled makes a speech-to-video avatar. You give it one reference frame and a speech track, and it animates the whole figure, not only the head, to match the speech. It does not create sound.

This is the distilled variant. It samples in 16 steps with guidance built into the weights, so no negative pass runs. The card says that is roughly 5 times faster than the 50-step base model.

Who it suits: Developers and creators who want talking-avatar video from a picture and a voice track, and who have a GPU. Use only images and recordings you have the right to use.

Industry | Who uses it | How they use it | Example
Marketing | Content teams | Make talking presenters from a still image | A product explainer with an animated host
Education | Course creators | Add a speaking character to lessons | A tutor figure that reads a script
Entertainment | Animators | Animate characters from a speech track | A cartoon character matching a voice line
Software | App developers | Add avatar replies to a product | A support avatar for an app

You can skip it if you have no GPU with large memory, or if you need a model that generates the voice itself.

Where to use it: It runs from the project's command line on a GPU machine. The card measured on one H100 and says generation can be split across several GPUs in one node.

• The kandinsky-s2v project, run with uv
• A GPU with large memory, with 57 GiB peak in the card's test
• Several GPUs on one node, to speed things up

Use case | Who | What they get back
Talking avatar video | Creators | A video of the reference figure speaking the track
Multilingual avatars | Studios | Animated speakers in other languages, per the project page
Long speech | Producers | Video generated in 5-second chunks

How to set it up: You need the project code from GitHub, the uv tool and a capable GPU. The first run downloads this checkpoint, the text encoders and the video autoencoder from the Hub.

• uv, a Python project tool
• A GPU with a lot of memory
• Disk space for the 4.2 GiB checkpoint plus the encoders

1. Get the kandinsky-s2v repository from GitHub and open its folder.
2. Install its dependencies with uv sync.
3. Run the generate command with your audio and reference image.

pip install -U huggingface_hub
hf download kandinskylab/Kandinsky-S2V-1.0-Lite-distill

How to use it: Run the generate command with the distilled config, your speech file, your reference image and an output path. The result is a 25 fps video in standard definition at the image's aspect ratio.

cd kandinsky-s2v
uv sync
uv run kandy-s2v generate \
    --config src/kandinsky_s2v/configs/s2v_lite_distill.yaml \
    --audio speech.wav \
    --image reference.png \
    --out out.mp4

Tool | What it does
uv sync | Installs the project's dependencies
kandy-s2v generate | Makes the avatar video from audio and an image
--config | Chooses the model setup, here the distilled Lite one
--audio and --image | Give the speech track and the reference frame

Example: You supply a still portrait and a spoken recording, and run the command. You get an out.mp4 in which the figure moves and speaks in time with your recording.

For newcomers and experts: If you are new, try one clear portrait and a short speech clip first. Longer speech is made in 5-second chunks, so start small to check the result.

If you know the field, it is latent diffusion with flow matching. Qwen2.5-Omni-7B supplies the text and audio conditions, a 2.3B-parameter DiT is the backbone with speech cross-attention on 13 of 32 blocks, and the HunyuanVideo 3D VAE encodes and decodes video.

Reference frame and speech track in
Qwen2.5-Omni encodes the conditions
DiT denoises in 16 steps
VAE decodes the frames
25 fps video out

Good to know: Most of the peak memory goes to decoding frames with the VAE. Decoding in tiles brings that peak down, though faint seams can show between tiles. The card lists these times on one H100 at 512 x 768, for each second of video made.

Title: Seconds to make one second of video on an H100
Standard run: 8.7 seconds
With compile: 7.1 seconds
Split across 4 GPUs: 3.8 seconds

• License: MIT
• Cost to run: none named, but it needs a high-memory GPU
• Data it reaches: your image and audio, plus checkpoints and encoders downloaded from the Hub on first run
• Not for: making sound, since the model only animates a figure to speech you supply

On Hugging Face, 13 November 2026

• Task: image to video
• Downloads in the last 30 days: 84
• Likes: 4
• License: mit
• Tags: kandinsky_s2v, speech-to-video, avatar, video-generation, kandinsky, image-to-video

Links

• Model: https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite-distill
• Files: https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite-distill/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download kandinskylab/Kandinsky-S2V-1.0-Lite-distill

Then follow the usage in its model card. Nothing in this kit runs by itself.