stable-diffusion-inpainting

stable-diffusion-inpainting

Generates photo-realistic images from text and fills masked areas of a picture, using Stable Diffusion with an extra mask input, in diffusers or AUTOMATIC1111. For designers, editors and developers.

Made by stable-diffusion-v1-5 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This model works in latent space and uses diffusion to turn a text prompt into a realistic-looking picture. It can also repaint part of an existing picture, which you mark with a mask. This repository is a mirror of an older repository that is now deprecated, and the card says it has no affiliation with RunwayML. The card names Robin Rombach and Patrick Esser as developers.

Who it suits: It suits people who want to change one region of an image from a text description, such as designers, photo editors and app developers.

Industry | Who uses it | How they use it | Example
Design | Illustrators and designers | Repaint one region of an image from a prompt | Putting a new subject into a masked spot of a scene
Photo editing | Retouchers | Replace or remove an object inside a mask | Swapping a masked item for something else
Software | App developers | Add region editing to an image application | Calling the diffusers pipeline from a backend
Research | Diffusion researchers | Study inpainting with extra mask input channels | Reading the card's training notes

If you never need to edit part of an image, you can skip it.

Where to use it:

• Python with the diffusers library
• AUTOMATIC1111, using the checkpoint file
• The original RunwayML GitHub repository, which the card marks as deprecated
• Windows, macOS or Linux with Python, for downloading the files

Use case | Who | What they get back
Region repainting | Designers | An image where only the masked area is redrawn
Object replacement | Editors | A picture with a masked object swapped for what the prompt describes
Scripted editing | Developers | A saved image file from a few lines of Python

How to set it up:

• Python and the Hugging Face hub package
• The diffusers library and torch, installed with pip
• An input image and a mask image, both as PIL images

1. Install the Hugging Face hub package.
2. Download the model files.
3. Install diffusers and torch.
4. Prepare your image and a mask, then run the pipeline from the next section.

pip install -U huggingface_hub
hf download stable-diffusion-v1-5/stable-diffusion-inpainting

How to use it: Load the pipeline, then pass a prompt, an image and a mask. In the mask, white marks the area to repaint and black marks what stays as it is. The card's snippet uses torch without importing it, so add that import yourself.

from diffusers import StableDiffusionInpaintPipeline

pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "sd-legacy/stable-diffusion-inpainting",
    revision="fp16",
    torch_dtype=torch.float16,
)
prompt = "Face of a yellow cat, high resolution, sitting on a park bench"
#image and mask_image should be PIL images.
#The mask structure is white for inpainting and black for keeping as is
image = pipe(prompt=prompt, image=image, mask_image=mask_image).images[0]
image.save("./yellow_cat_on_park_bench.png")

Tool | What it does
StableDiffusionInpaintPipeline | Loads the inpainting model in diffusers
pipe | Takes the prompt, the image and the mask, and returns images
mask_image | Marks the area to repaint in white and the area to keep in black
AUTOMATIC1111 | A web interface that can run the same checkpoint

Example: You give the pipeline a photo of a park bench, a mask over the spot where something should appear, and a prompt asking for a yellow cat sitting there in high resolution. The masked area is filled with the cat, and the rest of the picture stays as it was.

For newcomers and experts: Inpainting means redrawing only the part of a picture that you mask, guided by words. A good first try is to copy the snippet above, add a small photo and a mask, and write a short prompt.

Under the hood, the model started from Stable Diffusion v1.2 weights. Five input channels were added to the UNet. Four carry the masked image after encoding, one carries the mask, and all five begin with zero weights. During training, synthetic masks were generated, and in 25 percent of cases the mask covered everything. The text conditioning was dropped in 10 percent of samples to improve classifier-free guidance.

Prompt, image and mask go in
UNet with 5 extra input channels
Masked area is redrawn
Edited image comes out

Good to know:

Title: Training steps in thousands
Regular training: 595 k steps
Inpainting training: 440 k steps

• Inpainting training ran at a resolution of 512x512.
• The card lists English as the language.
• License: CreativeML OpenRAIL-M, so read its terms before use
• Cost to run: none named
• Data it reaches: none named beyond the images and prompts you give it, run locally
• Not for: none named in the part of the card shown here

On Hugging Face, 13 November 2026

• Task: text to image
• Library: diffusers
• Downloads in the last 30 days: 248,486
• Likes: 122
• License: creativeml-openrail-m
• Quantized versions: 1
• Fine-tunes: 6
• Tags: diffusers, stable-diffusion, stable-diffusion-diffusers, text-to-image

Links

• Model: https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-inpainting
• Files: https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-inpainting/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download stable-diffusion-v1-5/stable-diffusion-inpainting

Then follow the usage in its model card. Nothing in this kit runs by itself.