Drishya Tiny

Drishya Tiny

Turns a text prompt, or a prompt plus a first frame, into a 2-second 256×256 video clip with a 212M-parameter generator. A small research release that does best on waves, fire and snowy skies.

Made by udgamlabs on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Drishya Tiny is a small video generator with 212M parameters, trained from random weights by Udgam Labs. It pairs with a frozen Wan2.2 VAE and a frozen Flan-T5-Base text encoder. Give it a prompt, with or without a first frame, and it returns a short video that is 17 frames long at 8 fps and 256×256 pixels.

Who it suits: Researchers and curious builders who want to see how far a tiny model gets at video. The author labels it a research experiment and admits the output is brief, low resolution and frequently off target.

Industry | Who uses it | How they use it | Example
Research | Machine learning researchers | Study what a small generator learns from scratch | Compare results at different guidance settings
Education | Students | Learn how a text to video pipeline fits together | Run the Python example and read the config
Creative | Hobbyists | Make short nature clips for fun | Generate a clip of falling snow
Web development | Front-end developers | Try a model that runs on the device through WebGPU | Open the browser version and add a photo

People who need commercial use, long clips or realistic people can skip it, because the license is non-commercial and people do not come out well.

Where to use it:

• Python with PyTorch, on a CUDA GPU, Apple silicon or a CPU
• The command line, through the drishya module
• A desktop browser with WebGPU, such as Chrome or Edge, using the ONNX files
• A GPU demo on Hugging Face Spaces

Use case | Who | What they get back
Text to video | Hobbyists and researchers | A 2-second clip that follows the prompt loosely
Image to video | Creators | A clip that starts from your photo as its first frame
Browser trial | Anyone curious | A clip made on your own device, with nothing leaving it

How to set it up:

• Python, with PyTorch
• The drishya package from the project's GitHub repository, installed with the pip command given in the card
• This repository's files, which hold the generator only: model.safetensors and config.json

1. Install the Hugging Face hub tools with pip.
2. Download the model files with the hf command.
3. Install the drishya package using the card's pip command.
4. The frozen text encoder and VAE are fetched from their owners the first time the model loads.

pip install -U huggingface_hub
hf download udgamlabs/Drishya-Tiny

How to use it: Load the pipeline with from_pretrained, call it with a prompt, then write the frames out with save_mp4. For image to video, pass an image, which becomes the first frame and is cropped to 256×256 at the centre. There is also a command line form.

from drishya import Drishya, save_mp4

pipe = Drishya.from_pretrained("udgamlabs/Drishya-Tiny")   # also fetches Flan-T5-Base and the Wan2.2 VAE

# Text-to-video
prompt = "waves crashing on rocks at sunset"
frames = pipe(prompt, image=None, steps=50, cfg=5.0, shift=5.0, seed=0)
save_mp4(frames, "clip.mp4", prompt)

# Image-to-video: the image becomes the first frame (centre-cropped to 256×256) and is kept as given
from PIL import Image
prompt = "a boat drifting on a calm lake"
frames = pipe(prompt, image=Image.open("lake.jpg"), seed=0)
save_mp4(frames, "boat.mp4", prompt)

python -m drishya "waves crashing on rocks at sunset" --out clip.mp4
python -m drishya "a boat drifting on a calm lake" --image lake.jpg --out boat.mp4

Tool | What it does
Drishya.from_pretrained | Loads the generator and fetches the frozen text encoder and VAE
save_mp4 | Writes the video, with an AI-generated label and provenance metadata
python -m drishya | Runs the same pipeline from the command line
--image | Gives a photo to use as the first frame
--out | Sets the output file name

Example: You ask for fire burning in a dark fireplace, since fire is one of the subjects the card says works best. You get a short silent mp4 of about 2 seconds at 256×256, small and soft, and it may not match every detail of the prompt.

For newcomers and experts: Start with scenery, such as water, sky or snow, with one subject, no text and no faces. Keep the AI-generated label and the metadata when you share a clip. The card asks for both.

Under the hood, the generator is a VideoDiT with width 768, depth 16 and 12 heads. The card says defaults of 50 Euler steps, guidance 5 and shift 5 produce its benchmark numbers. Guidance 3 scores better on FVD, 373 against 432, but follows prompts a little less well. Prompts are cut at 77 tokens.

A prompt, and optionally a first frame, go in
Flan-T5-Base encodes the prompt
The 212M generator denoises the video latents
The Wan2.2 VAE decoder turns them into frames
save_mp4 writes the labelled clip

Good to know:

• Quality: Expect brief, low resolution videos that miss the prompt fairly often. Water, flames and wintry skies come out best, while people come out badly.
• Label: save_mp4 stamps a small AI-generated tag on every frame and records the prompt, seed and model in the file. The card asks you to keep both when sharing.
• Browser: The browser version downloads 665 MB once and uses a lighter decoder, so its clips look softer.
• Size: The whole system is about 877M parameters for text to video and about 1.03B for image to video.
• License: CC BY-NC 4.0, so non-commercial use only. The frozen parts keep their own licences.
• Cost to run: none named. It uses a CUDA GPU if there is one, then Apple silicon, then the CPU.
• Data it reaches: Your prompts and photos. The browser version keeps them on your device, and the Python package fetches the frozen parts from their owners.
• Not for: Commercial use, readable text or realistic people. The card also links a use policy.

On Hugging Face, 13 November 2026

• Task: text to video
• Library: pytorch
• Parameters: 212M
• Weights: about 848 MB in F32; about 106 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 163
• Likes: 6
• License: cc-by-nc-4.0
• Tags: pytorch, onnx, safetensors, udgamlabs, text-to-video, image-to-video, diffusion-transformer, rectified-flow, small-model

Links

• Model: https://huggingface.co/udgamlabs/Drishya-Tiny
• Files: https://huggingface.co/udgamlabs/Drishya-Tiny/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download udgamlabs/Drishya-Tiny

Then follow the usage in its model card. Nothing in this kit runs by itself.