Matrix-Game 4.0

Matrix-Game 4.0

Redraws the frames of a running game as clean low-poly video, with one renderer per scene: Ancient Egypt, City Racing and a medieval castle. Built for game developers and video model researchers.

Made by Skywork on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Matrix-Game 4.0 is a set of video renderers, each trained for one scene: Ancient Egypt, City Racing and a medieval castle. A running game decides gameplay, physics and input, and the model redraws its frames as clean low-poly video. The heads-up display and the sound stay separate layers that line up with the frames.

Who it suits: Game developers and researchers who work on neural rendering of game scenes are the main fit. Students of generative video can also read it as a worked example of a model tied to a game.

Industry | Who uses it | How they use it | Example
Games | Game developers | Redraw a game's output as clean low-poly video | Render the City Racing scene from the game's frames
Research | Video model researchers | Study renderers that are specialised for one scene | Compare the one-step and two-step presets
Education | Students of generative video | Read a worked example of a model tied to a game engine | Open the release manifests and configuration files

If you want a drop-in Diffusers pipeline, a guaranteed frame rate or a model for any scene, you can skip it.

Where to use it: It runs inside the project's own inference source code, not inside Diffusers or Transformers.

• The Matrix-Game 4.0 inference source, which the weights require
• A single GPU with CUDA graphs for the realtime preset
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
First-person scenes | Game developers | Clean low-poly frames for the Ancient Egypt and castle scenes
Racing scene | Game developers | Frames for City Racing, with fixed or per-chunk engine captions
Speed and quality tests | Researchers | A quality preset and a realtime preset to compare

How to set it up: You need Python, a terminal, a GPU and room for large checkpoints.

• Python with pip, on Windows, macOS or Linux
• A GPU, though the card promises no memory fit for consumer cards
• Space for scene checkpoints, each with 5,001,016,512 BF16 parameters

1. Install the Hugging Face hub package.
2. Download the repository files, or use the release's manifest downloader with a pinned revision to fetch only the scenes you pick.
3. Get the Matrix-Game 4.0 inference source, because the weights do not run without it.

pip install -U huggingface_hub
hf download Skywork/Matrix-Game-4.0

The manifest downloader fetches the shared T5 encoder, tokenizer and Wan2.2 VAE itself and checks each file's length and SHA256. Do not guess the local folder layout from the Hub folder names, since the manifest maps them to runtime paths.

How to use it: The game supplies input and physics, and the inference source feeds its clean RGB frames to the model. Output is 480x864 RGB, and one latent chunk covers four frames after startup. Pick a preset to trade quality against cost.

Tool | What it does
Quality preset | Uses Wan VAE encoding and decoding with two denoising steps and the native DiT
Realtime preset | Uses TAEHV encoding and decoding with one step, an optimized single-GPU DiT and CUDA graphs
base/taew2_2.pth | Holds the full streaming fast codec, and a standalone decoder file cannot replace its encoder

The castle and Ancient Egypt scenes use an independent identity sink, while Racing uses your supplied image as output frame zero with an absolute-position sink. Do not swap these policies.

Example: You start the City Racing scene with a supplied first image and a drive through the game's map. The model treats the image as output frame zero, skips denoising on it, and renders the following frames at 480x864 RGB. The HUD and the sound arrive from their own layers.

For newcomers and experts: This is not a model that makes video from a text prompt. It redraws frames from a game that is already running. A good first try is to download the files with the two commands above, then open config.json and the releases folder to see the scene list and manifests.

Each scene has its own checkpoint, and the record lists alibaba-pai/Wan2.2-Fun-5B-Control as the base model. The cache holds 21 latent frames including the sink, with CFG set to 1 and a scheduler shift of 5. The quality and realtime presets trade image quality for speed and cost.

The running game supplies input and physics
Its clean RGB frames go to the model
The model redraws each chunk of frames
HUD and sound join as separate layers

Good to know: Each model is tuned to the scenes and camera setups that ship with it. Other people have published one fine-tuned version of it.

• Limits: new shapes, fast camera moves, objects hiding each other and long sessions can make the look drift or the geometry go wrong
• Gameplay: the game keeps control of collisions and play, so the rendering never alters them
• Not drop-in: the weights are not Diffusers pipelines, and training code and data are not included
• No guarantees: the card promises no frame rate, no memory fit and no exact scene copy, and it claims no benchmark
• License: Apache 2.0 for the scene weights, sink images and game assets from Skywork AI
• Third parties: the shared Wan and umT5 files keep their own Apache 2.0 terms, and TAEHV is under the MIT license
• Audio credits: some racing sounds carry CC0 or CC BY 3.0 terms, so keep the credits file that ships with them
• Cost to run: none named. It needs a GPU, and the card gives no memory figure
• Data it reaches: files downloaded from Hugging Face and checked against manifests. The card names no account or web service
• Not for: training, since no training code or data is included, or scenes it was not built for

On Hugging Face, 13 November 2026

• Task: video to video
• Formats: Safetensors
• Downloads in the last 30 days: 1,150
• Likes: 62
• License: apache-2.0
• Based on: alibaba-pai/Wan2.2-Fun-5B-Control
• Fine-tunes: 1
• Tags: safetensors, matrix-game-4.0, causal-video, game-rendering, video-to-video

Links

• Model: https://huggingface.co/Skywork/Matrix-Game-4.0
• Files: https://huggingface.co/Skywork/Matrix-Game-4.0/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Skywork/Matrix-Game-4.0

Then follow the usage in its model card. Nothing in this kit runs by itself.