sd-vae-ft-mse-original

sd-vae-ft-mse-original

Replaces the autoencoder in Stable Diffusion with a fine-tuned decoder that reconstructs images, especially faces, more smoothly. Suits people running the original CompVis Stable Diffusion code who want better decoded images.

Made by stabilityai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: sd-vae-ft-mse-original holds an improved autoencoder for Stable Diffusion, published by Stability AI. An autoencoder turns images into a compact form and back, and Stable Diffusion uses its decoder to draw the final picture. This one is the ft-MSE version, with only the decoder fine-tuned, so it can replace the existing autoencoder directly.

Who it suits: People who run the original CompVis Stable Diffusion code and want cleaner decoded images, under the MIT license.

Industry | Who uses it | How they use it | Example
Creative work | Artists | Swap the autoencoder in a Stable Diffusion setup | Get smoother faces in generated pictures
Software | Developers | Test the decoder in an image generation pipeline | Compare outputs with the original autoencoder
Research | ML researchers | Study autoencoder fine-tuning and reconstruction losses | Reproduce the reported scores
Tooling | Tool builders | Ship a better default decoder | Replace the autoencoder file in a toolkit

Skip it if you use the diffusers library, since the card says these weights target the original CompVis codebase and points diffusers users to a different repository.

Where to use it: The card says to use it with the original CompVis Stable Diffusion code.

• The original CompVis Stable Diffusion codebase
• Windows, macOS or Linux, with Python for the download
• Existing Stable Diffusion checkpoints, since only the decoder changed

Use case | Who | What they get back
Decoder swap | Artists | Decoded images with smoother output
Face reconstruction | Developers | Better reconstruction of human faces than the original
Benchmarking | Researchers | Reconstruction scores to compare with other autoencoders

How to set it up: You need Python and a Stable Diffusion setup that uses the original code.

• Python with pip
• The original CompVis Stable Diffusion codebase
• Space for the checkpoint file

1. Install the Hugging Face hub tools with pip.
2. Download the repository with the hf command.
3. Point your Stable Diffusion setup at the downloaded autoencoder checkpoint.

pip install -U huggingface_hub
hf download stabilityai/sd-vae-ft-mse-original

How to use it: Use the downloaded checkpoint where the original autoencoder file goes in your Stable Diffusion setup. The card says it works as a drop-in replacement and gives no extra commands, so you run generation as before.

Tool | What it does
hf download | Fetches the repository to your computer
ft-MSE checkpoint | The autoencoder file that replaces the original one
CompVis Stable Diffusion code | Loads the autoencoder during generation

Example: You generate a portrait with your usual Stable Diffusion prompt, once with the original autoencoder and once with this one. The second picture should decode with smoother detail, and you compare the two by eye.

For newcomers and experts: If you are new to Stable Diffusion, think of the autoencoder as the last step that turns the model's compressed picture into pixels. Swapping it changes how sharp or smooth the result looks, and leaves the rest of the model alone.

If you know the field, two kl-f8 autoencoders were fine-tuned from the original on a 1:1 mix of LAION-Aesthetics and a human-image subset. The ft-EMA version came first, and ft-MSE resumed from it for another 280 thousand steps with a loss that stresses MSE, which gives smoother outputs. Only the decoder was tuned.

Start from the original kl-f8 autoencoder
Fine-tune the decoder, ft-EMA
Resume with an MSE-heavy loss, ft-MSE
Swap the file into Stable Diffusion
Decode images with smoother output

Good to know: On the COCO 2017 test, rFID fell from 4.99 for the original to 4.42 for ft-EMA and 4.70 for ft-MSE, where lower is better. ft-MSE had the highest PSNR on both test sets the card reports.

Title: rFID on COCO 2017, lower is better
Original: 4.99
ft-EMA: 4.42
ft-MSE: 4.70

• License: MIT
• Cost to run: none named, apart from your own hardware
• Data it reaches: the checkpoint on your computer, and nothing else the card names
• Not for: the diffusers library, since the card says these weights are meant for the original CompVis code

On Hugging Face, 13 November 2026

• Task: text to image
• Downloads in the last 30 days: 6
• Likes: 1,447
• License: mit
• Tags: stable-diffusion, stable-diffusion-diffusers, text-to-image

Links

• Model: https://huggingface.co/stabilityai/sd-vae-ft-mse-original
• Files: https://huggingface.co/stabilityai/sd-vae-ft-mse-original/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download stabilityai/sd-vae-ft-mse-original

Then follow the usage in its model card. Nothing in this kit runs by itself.