GREAT Outpainting with ControlNet!

Sebastian Kamph
12 Jun 202310:11

TLDRIn this video, the creator demonstrates how to use ControlNet and the inpainting model to expand images. They walk through step-by-step instructions on resizing both vertically and horizontally, and improving details through image-to-image processing. By adjusting denoising strength, users can fine-tune changes while keeping key elements intact. The video also shows techniques to sharpen images for added clarity, and how to apply this process to human portraits. Overall, it's a comprehensive guide on enhancing images with ControlNet for outpainting and refining visual elements.

Takeaways

  • ๐ŸŒŸ ControlNet helps expand images easily and effectively.
  • ๐Ÿ–ผ๏ธ The process involves using an in-painting model to expand specific areas of an image, such as a Fantasy Island with a castle.
  • ๐Ÿ“ It's crucial to set the resize mode to 'resize and fill' to avoid stretching or cropping the image.
  • ๐ŸŒฅ๏ธ Outpainting can add new elements like clouds, water, and stones while preserving the original composition.
  • ๐ŸŽจ You can improve the image quality by using image-to-image features, adjusting denoising strength between 0.4 and 0.6 for optimal results.
  • ๐Ÿ”ง Sharpening the image using tools like photop.com before reapplying image-to-image can enhance detail and contrast.
  • ๐Ÿ‘ฉ ControlNet works well with generating realistic portraits, and you can expand images vertically and horizontally for people as well.
  • ๐Ÿ’ก Outpainting vertically and horizontally needs to be done separately to avoid stretching the image.
  • โœ๏ธ You can customize prompts to control what gets added during outpainting, especially when working with faces or specific features.
  • ๐Ÿ‘ The workflow provides flexibility, but experimenting with settings and prompts is key to achieving the best results.

Q & A

  • What is the primary focus of this video?

    -The video demonstrates how to expand images using ControlNet and the inpainting model.

  • What is the image used in the video?

    -The image used is a Fantasy Island with a castle on it.

  • What settings are necessary to expand the image correctly?

    -You need to change the resize mode to 'resize and fill' to avoid stretching or cropping the image. This will allow proper expansion while filling the empty spaces.

  • What happens if you use just 'resize' without 'fill'?

    -The image will be stretched vertically or horizontally, leading to distortion instead of proper expansion.

  • What is the purpose of using DPM++ 2M Keras with 20 steps in this process?

    -DPM++ 2M Keras is the preferred model for generating images with more control, and 20 steps are used to achieve a balance between speed and detail.

  • How can you improve the quality of the expanded image?

    -You can improve the quality by sending the expanded image to 'image to image', adjusting the denoising strength, and applying filters such as sharpening or unsharp masking.

  • What is the recommended range for denoising strength in 'image to image'?

    -The recommended denoising strength is between 0.4 and 0.6, depending on how much of the original image you want to retain while adding new details.

  • How does sharpening the image before sending it to 'image to image' affect the result?

    -Sharpening the image can make the final output crisper, adding more contrast and detail to areas that may have appeared blurry.

  • Why can't you expand the image vertically and horizontally at the same time?

    -Expanding in both directions at once would stretch the image and not add meaningful information. Itโ€™s better to expand in one direction at a time for better results.

  • Can this technique be used for images of people?

    -Yes, the method works for images of people as well, and the video demonstrates this by expanding a portrait image with a similar process.

Outlines

00:00

๐ŸŽจ How to Expand Your Image with Control Net

In this introduction, the presenter explains how to expand images using Control Net with the in-painting model. He starts with an image of a fantasy island and shows how to load it into the Control Net interface, enabling in-painting preprocessing. The key step is setting the resize mode to 'resize and fill' rather than just 'resize,' to avoid stretching or cropping the image. The presenter begins by expanding the image vertically, adjusting parameters like the Euler A sampler to DPM++ 2M Keras, using 20 steps, and maintaining default settings for negative and digital oil painting styles. He highlights the importance of selecting the right resizing mode to avoid image distortion.

05:03

๐Ÿ“ Expanding Horizontally and Fine-Tuning Image Details

Here, the focus shifts to horizontal expansion, with the same settings applied. As the image expands, new details like clouds, water, and rocks appear. The presenter acknowledges that expanding from a small image (512x512) results in less detail. To improve, he suggests transferring the expanded image to the 'image-to-image' section, where more details can be added using styles like 'Fantasy Island' and adjusting denoising strength between 0.4 and 0.6. The difference between low and high denoising strength is demonstrated, with higher values introducing more changes and details to the image, such as altering the castle and bridge slightly.

10:04

๐Ÿ”ง Sharpening and Refining the Expanded Image

This section provides a trick for improving sharpness and detail in the image by using an external tool like photop.com. By applying a filter called 'unsharp mask' with specific settings, the presenter sharpens the image slightly, which can then be used as input for the 'image-to-image' process. He shows how sharpening the image before re-rendering results in more contrast and clarity in fine details, particularly visible in terrain and water. The presenter compares the initial and improved versions side-by-side, emphasizing how these small changes can enhance the overall quality of the expanded image.

๐Ÿ‘ฉ Generating and Expanding Realistic Portraits

Shifting focus to human portraits, the presenter explains how the same workflow can be applied to generate and expand images of people. He demonstrates this with a series of generated female portraits, selecting one with a hat for expansion. Using Control Net and the same settings (resize and fill), he first expands vertically and then horizontally, noting that it's necessary to expand one direction at a time to avoid stretching. This method generates new parts of the image seamlessly, adding to the portrait while maintaining the original composition.

๐Ÿ› ๏ธ Tips for Outpainting Workflow Optimization

In this final instructional part, the presenter answers potential questions about expanding images in multiple directions at once. He explains why his preferred workflow involves expanding one axis at a time and encourages viewers to explore alternative methods, such as 'Poor Man's Outpainting.' He shows examples of two expanded portrait images, discussing minor imperfections in one but ultimately being satisfied with the results. He wraps up by suggesting that users be mindful of prompts when outpainting to ensure the best results and invites viewers to share better workflows in the comments.

๐Ÿ‘ Final Thoughts and Call to Action

In the conclusion, the presenter thanks viewers for watching and encourages them to like and subscribe to the channel if they found the tutorial helpful. He maintains a lighthearted tone, stating that viewers are free to do as they please. The video closes with a warm goodbye, leaving the audience with the option to apply these techniques in their own image-editing workflows.

Mindmap

Keywords

๐Ÿ’กControlNet

ControlNet is a tool used to manipulate and control the expansion of images in AI art generation. In this video, it is employed to help extend images with precision, specifically when creating additional content around an existing image, such as expanding a scene in a fantasy landscape.

๐Ÿ’กInpainting

Highlights

How to expand an image easily using ControlNet and in-painting models.

ControlNet and in-painting model overview, including when to use each.

Setting up ControlNet with the in-painting preprocessor for image expansion.

Explanation of resizing and filling mode to avoid image stretching or cropping.

Adding vertical expansion to the image by increasing the height.

How to improve detail when expanding small images, such as 512x512 resolutions.

Expanding images horizontally and how ControlNet handles the expansion with added details.

Using image-to-image technique to refine and improve the expanded composition.

Adjusting denoising strength in image-to-image, recommending values between 0.4 to 0.6.

How higher denoising introduces more changes and details to the expanded image.

Sharpening expanded images using external tools like Photopea with unsharp mask filter.

Handling blurry images and adding crispness with final sharpening.

Generating realistic photo portraits using ControlNet for face expansions.

Expanding portrait images by adjusting height and width separately to avoid distortion.

Expanding in multiple directions and maintaining control over what new information is added.