Primal VAE

Primal VAE

Provides small image autoencoders that compress pictures to one or two latent channels and need no scaling factor, then patch up to match the Flux or Stable Diffusion latent shapes. For people building diffusion models.

Made by ostris on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Primal VAE is a small image autoencoder for diffusion work. It shrinks an image only 2x or 4x in each direction and keeps just one or two channels, instead of using deep compression. You then patch the latent up to the shape a diffusion transformer expects.

The latent is trained to have the same spread as noise. That means you skip the scaling factor, the shift factor and any latent normalization. The flow is encode, patch, diffuse.

Who it suits: It suits researchers and developers who train or adapt diffusion models and want a light VAE that fits the latent shapes of Flux and Stable Diffusion models.

Industry | Who uses it | How they use it | Example
AI research | Diffusion researchers | Try a VAE with no scaling factor | Train a model on its unit-normal latents
Software | ML engineers | Swap it in for the VAE in a pipeline | Replace the Flux.1 VAE latent
Image tools | Tool builders | Encode and decode images with a small network | Round-trip a picture through it
Training | Fine-tuners | Backpropagate through decode for a pixel loss | Use gradient checkpointing

If you only want to generate pictures with a finished app, you can skip it, since this is a building block for model work.

Where to use it: It loads through the diffusers library in Python, on a GPU machine, and the card shows it on CUDA.

• Python scripts that use diffusers
• Diffusion transformer training and research code
• Pipelines that use Flux.1, Flux.2 or Stable Diffusion latent shapes

Use case | Who | What they get back
Match a Flux.1 latent | Pipeline builders | A 16-channel latent at 8x
Match a Flux.2 latent | Pipeline builders | A 32-channel latent at 8x
Match an SD 1.5 or SDXL latent | Pipeline builders | A 4-channel latent at 8x

How to set it up: You need Python with pip, the diffusers library, PyTorch and a GPU. The card loads the model with trust_remote_code turned on, so read the repository code before you run it.

• Python with pip
• diffusers and PyTorch
• A CUDA GPU, as in the card's examples

1. Install the Hugging Face hub package.
2. Download the repository, or let diffusers fetch it by id.
3. Load a variant with AutoModel and move it to the GPU.

pip install -U huggingface_hub
hf download ostris/primal_vae

How to use it: You pick a variant, encode a batch of images, and decode the latent back. Image sides must be multiples of 16 for the 2x models and 32 for the 4x1 model.

import torch
from diffusers import AutoModel

vae = AutoModel.from_pretrained("ostris/primal_vae", variant="2x1", trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")

x = ...                      # [B, 3, H, W] in [-1, 1], H and W multiples of vae.image_multiple (16, or 32 for 4x1)
z = vae.encode(x)            # [B, 1, H/2, W/2], ~N(0, 1). No .latent_dist, no sampling.
x_hat = vae.decode(z)        # [B, 3, H, W]

Variant | What it gives | Shape it can match
2x1 | One channel at 2x | The Flux.1 VAE latent
2x2 | Two channels at 2x | The Flux.2 VAE latent
4x1 | One channel at 4x | The SD 1.5 and SDXL VAE latent

Example: You load the 2x1 variant, encode a batch of images, and call patchify with compression 8. You get tokens with the Flux.1 latent shape, ready to drop into a Flux pipeline, and decode returns the picture.

For newcomers and experts: If you are new, load the 2x1 variant, encode one picture and decode it again to see the round trip. That shows what the model does before you change a pipeline.

If you know the field, encoder and decoder are mirrored convolutional U-Nets using RMSNorm and no attention. With a little overlap, a tile gives the same result as the full image. It has no posterior, KL term or quant convolution.

Image is pixel-unshuffled
Encoder gives a unit-normal latent
Latent is patched up to 8x
Diffusion model works on tokens

Good to know: The 2x models have about 80M parameters and the 4x1 model has about 122M. Variant names pick the weight file, and a fine-tuned checkpoint can be loaded by its file name.

• License: apache-2.0
• Cost to run: none named, though it needs a GPU for the card's examples
• Data it reaches: files you download, and the images you pass in
• Not for: loading without care, since the card sets trust_remote_code to true

On Hugging Face, 13 November 2026

• Library: diffusers
• Downloads in the last 30 days: 1,268
• Likes: 49
• License: apache-2.0
• Quantized versions: 1
• Tags: diffusers, autoencoder, latent-diffusion

Links

• Model: https://huggingface.co/ostris/primal_vae
• Files: https://huggingface.co/ostris/primal_vae/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ostris/primal_vae

Then follow the usage in its model card. Nothing in this kit runs by itself.