ComfyUI ControlNet Tutorial: Control Composition with Line Art, Poses, and Depth

It's difficult to precisely control "a person's pose," "a building's perspective," or "the composition of an image" with prompts alone. ControlNet lets you provide a reference image and extract structural information from it (lines, poses, depth) so the generated image follows that structure.

Common control types

What you need

1. ControlNet model: Place it in models/controlnet/. It must match the base model version (SD 1.5, SDXL, and Flux each have different ControlNet models). There are also "Union" models that support multiple control types in a single file.
2. Preprocessor: Converts a reference photo into the image ControlNet needs (for example, turning a photo into lines or a skeleton). Canny has a built-in node; other types usually require the comfyui_controlnet_aux custom node (see Custom nodes and Manager).

Build a workflow (using Canny)

[Load Image] ─▶ [Canny] ─IMAGE────────────────┐
                                              ▼
[CLIP Text Encode Positive] ─CONDITIONING─▶ [Apply ControlNet] ─positive─▶ [KSampler]
[CLIP Text Encode Negative] ─CONDITIONING─▶                    ─negative─▶
[Load ControlNet Model] ─CONTROL_NET──▶
                         vae ─────────▶

Steps:

1. Start with the text-to-image workflow.
2. Add Load Image and load a reference image.
3. Add a Canny node and connect the reference image. Run it once and use Preview Image to check the extracted lines.

4. Add Load ControlNet Model and select a Canny model.
5. Add Apply ControlNet:
   • Connect positive and negative to the outputs of the two CLIP Text Encode nodes
   • Connect control_net to Load ControlNet Model
   • Connect image to the Canny output
   • Connect vae to the model's VAE (required by some ControlNet models)
6. Connect the positive and negative outputs of Apply ControlNet to KSampler.
7. Set the dimensions of Empty Latent Image to the same aspect ratio as the reference image.

Parameters

Tip: Set end_percent to 0.5–0.7. ControlNet determines the composition during the first half, then the model has room to create details during the second half. The result is usually more natural.

OpenPose: Control a person's pose

1. Use the DWPose Estimator or OpenPose Pose preprocessor (from controlnet_aux) to extract a skeleton from a photo of a person.
2. Use Preview Image to check that the skeleton is correct.
3. Connect an OpenPose ControlNet model.

The generated person will have the same pose as in the photo, while the prompt determines their appearance, clothing, and setting.

Depth: Preserve spatial structure

Use a preprocessor such as Depth Anything to generate a depth map. This works well for:

• Redesigning an interior photo in a different decor style
• Preserving the perspective of a building or scene while changing the season, time of day, or art style

Combine multiple ControlNets

Chain multiple Apply ControlNet nodes: connect the positive and negative outputs of the first to the inputs of the second. For example, use OpenPose to control the pose and Depth to control the scene. Lower each strength moderately to prevent them from conflicting.

Common problems

Updated October 12, 2026