ControlNet v1.1 depth (SD 1.5)

ControlNet v1.1 depth (SD 1.5)

Steers Stable Diffusion 1.5 with a depth map so generated images keep the layout of a source picture. ControlNet v1.1 in diffusers format, for artists and developers who want control over composition.

Made by lllyasviel on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is the depth version of ControlNet v1.1, converted to the diffusers format. ControlNet adds an extra condition to a diffusion model, and here the condition is a depth image. You pair it with Stable Diffusion v1-5, and your prompt then paints new content inside the depth layout you supply.

Who it suits: Artists, designers and developers who want to control the composition of generated images, not just the subject. The author recommends Stable Diffusion v1-5, and notes that other diffusion models such as dreamboothed ones may work experimentally.

Industry | Who uses it | How they use it | Example
Design | Designers | Keep a scene's depth layout while changing its style | Re-rendering a room photo as a different interior
Game art | Concept artists | Turn rough depth views into varied images | Several moods from one depth layout
Software | Developers building image tools | Add depth control to a diffusers pipeline | A depth-guided generation feature in an app
Research | Diffusion researchers | Study conditional control of diffusion models | Comparing outputs with and without the condition

You can skip it if you do not use Stable Diffusion 1.5 or diffusers, since this checkpoint was trained on that base.

Where to use it:

• Python projects that use the diffusers library
• Stable Diffusion v1-5 pipelines
• Windows, macOS or Linux with Python installed

Use case | Who | What they get back
Depth-guided image generation | Artists | New images that follow a depth map
Layout-preserving restyling | Designers | A changed look with the same spatial layout
Conditional control experiments | Researchers | Outputs to compare across conditions

How to set it up:

• Python on Windows, macOS or Linux
• The huggingface_hub package, then diffusers, transformers and accelerate
• Room for about 1.4 GB of weights in F32, plus Stable Diffusion v1-5

1. Install the Hugging Face client.
2. Download the model files to your computer.
3. Install diffusers and its related packages.
4. Load the checkpoint next to Stable Diffusion v1-5 in a ControlNet pipeline.

pip install -U huggingface_hub
hf download lllyasviel/control_v11f1p_sd15_depth

pip install diffusers transformers accelerate

How to use it: Make a depth image from your input picture with a depth-estimation pipeline, turn it into a three-channel control image, then load the ControlNet and pair it with Stable Diffusion v1-5. Add your prompt and run the pipeline. The card's full code runs on a half-precision setting.

Tool | What it does
depth-estimation pipeline | Makes a depth map from your input image
ControlNetModel | Loads this depth checkpoint
StableDiffusionControlNetPipeline | Combines the ControlNet with Stable Diffusion v1-5
UniPCMultistepScheduler | Scheduler the card's example imports for sampling

Example: You supply a photo of a lecture hall and the prompt "a grand library with warm lamps". The depth estimator reads the room's layout, and the pipeline paints a library that keeps the same rows and distances.

For newcomers and experts: If you are new, think of the depth map as a guide for where things sit near and far, with your prompt filling in what they look like. A first try is one photo, one prompt and the default settings.

If you know the field: ControlNet learns task-specific conditions end to end and attaches to a pretrained diffusion model. The authors report that it trains about as fast as fine-tuning and works even with small datasets.

Input image
Depth estimator
Depth control image
ControlNet with Stable Diffusion
New image

Good to know: The model details list English as the language and describe a text-to-image diffusion model. The checkpoint was trained on Stable Diffusion v1-5.

• Base: Stable Diffusion v1-5 is the recommended partner
• Authors: Lvmin Zhang and Maneesh Agrawala
• License: CreativeML OpenRAIL M
• Cost to run: none named
• Data it reaches: model files downloaded to your computer, plus the images you feed it
• Not for: use outside the CreativeML OpenRAIL M terms, which the source does not spell out

On Hugging Face, 13 November 2026

• Task: image to image
• Library: diffusers
• Parameters: 361M
• Weights: about 1.4 GB in F32; about 181 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 22,041
• Likes: 70
• License: openrail
• Based on: runwayml/stable-diffusion-v1-5
• Tags: diffusers, safetensors, controlnet, stable-diffusion, controlnet-v1-1, image-to-image

Links

• Model: https://huggingface.co/lllyasviel/control_v11f1p_sd15_depth
• Files: https://huggingface.co/lllyasviel/control_v11f1p_sd15_depth/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download lllyasviel/control_v11f1p_sd15_depth

Then follow the usage in its model card. Nothing in this kit runs by itself.