PAN 2

PAN 2

A 29B world model from IFM at MBZUAI that turns a scene and described actions into a video of up to 20 seconds, so an agent can preview outcomes. For researchers and agent builders with a large GPU.

Made by IFM on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: PAN 2 is a world model from the Institute of Foundation Models at MBZUAI. It predicts how a scene changes after actions that you describe in text. You give the current scene as text, with an optional image, plus the actions, and it returns a video of the result that runs up to 20 seconds. This repository holds the 29B diffusion backbone in BF16.

Who it suits: It suits researchers and agent builders who want to see the likely outcome of an action before choosing it, and video makers who want short clips from a described scene.

Industry | Who uses it | How they use it | Example
AI research | Researchers | Study world models and simulative reasoning | Compare candidate actions by their simulated outcomes
Agents | Agent builders | Let an agent preview outcomes before it acts | Simulate each plan and pick the best-looking result
Media | Video creators | Make a short clip from a first frame and a description | A clip of up to 20 seconds from one image
Simulation | Developers | Prototype how a scene may change after an action | A scene after a character carries out a step

If you want a light video tool for a small GPU, you can skip it, since the weights alone are about 58.4 GB.

Where to use it:

• Python with conda, using the generate.py script from the card
• Diffusers and vLLM, which the card says also support the model
• Machines with NVIDIA GPUs, since the setup uses CUDA builds of PyTorch

Use case | Who | What they get back
Text to video | Creators | A video made from a structured text description
Image to video | Creators | A video that starts from your first frame
Action simulation | Agent builders | A video of the outcome of an action you describe

How to set it up: You need these first:

• Python and conda, with Python 3.14 as in the card
• A CUDA GPU and disk space for about 58.4 GB of BF16 weights
• PyTorch 2.12.1 and torchvision 0.27.1 built for CUDA
• The text encoder Qwen3.5-9B and the Wan 2.2 VAE, which download from their own repositories

1. Install the Hugging Face hub tool and download the transformer folder.
2. Convert the weights once to the layout that generate.py reads.
3. Create the conda environment and install the libraries.
4. Download the frozen text encoder and VAE.
5. Point CKPT_PATH at the converted transformer folder and run generate.py.

pip install -U huggingface_hub
hf download IFM/PAN2 --include "transformer/*" --local-dir /path/to/PAN2
python scripts/convert_checkpoint.py to-native \
    --input /path/to/PAN2/transformer --output <DIR>/transformer

conda create -n pan python=3.14 -y && conda activate pan
pip install --pre "flash-attn-4[cu13]==4.0.0b29" --no-build-isolation
pip install "quack-kernels[cu13]==0.6.4"
pip install -r requirements.txt

export HF_HOME=YOUR_HF_PATH
hf download Qwen/Qwen3.5-9B
hf download Wan-AI/Wan2.2-TI2V-5B-Diffusers

How to use it: Describe the scene and the actions in the structured text form the card expects, add a first frame if you have one, and run generate.py. The part of the card given here ends before the generate command, so read the full card for the exact call.

Tool | What it does
generate.py | The script the card uses to make videos
convert_checkpoint.py | Converts the released weights to the layout generate.py reads
Diffusers | Also supports PAN 2, per the card
vLLM | Also supports PAN 2, per the card

Example: You give a picture of a cup on a table and write that a hand pushes it off the edge. PAN 2 returns a video of up to 20 seconds that shows what follows. The card does not show this exact run, so treat it as an expectation.

For newcomers and experts: A world model predicts what happens next in a scene. A good first step is to read the card's overview and check that your GPU can hold the weights before you download anything.

The card describes two parts. A large language model first reads the scene and the actions, then drafts a structured text description of what the world looks like next. A latent diffusion model then predicts the video that matches it, and the Wan 2.2 decoder turns that into frames. The released checkpoint is the 29B diffusion part, which the card says learned from scratch using videos numbering in the hundreds of millions.

Describe the scene and the actions
The language model writes the next world state
The diffusion model predicts the video latents
The decoder renders the video

Good to know: The weights are large, and the 4-bit figure is only Hugging Face's own estimate.

Title: Weight size in GB
BF16 weights: 58.4 GB
4-bit estimate: 14.6 GB

• Output: video at 704x1248 and 24 fps, up to 20 seconds
• Tasks: one checkpoint covers text to video and image to video
• Conversion: the weights ship in the diffusers layout, so convert them once for generate.py
• Separate downloads: the text encoder and VAE come from other repositories
• License: Apache 2.0
• Cost to run: none named, you supply the GPU
• Data it reaches: the text and images you give it, and the extra models you download

On Hugging Face, 13 November 2026

• Task: image text to video
• Library: diffusers
• Parameters: 29.2B
• Weights: about 58.4 GB in BF16; about 14.6 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 10
• Likes: 83
• License: apache-2.0
• Languages: en
• Fine-tunes: 1
• Tags: diffusers, safetensors, world-model, world-simulation, text-to-video, image-to-video, image-text-to-video, diffusion-transformer

Links

• Model: https://huggingface.co/IFM/PAN2
• Files: https://huggingface.co/IFM/PAN2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download IFM/PAN2

Then follow the usage in its model card. Nothing in this kit runs by itself.