Civitai AI Video & Animation // Making Depth Maps for Animation // 3.28.24

17 Apr 202481:31

TLDRIn the Civitai AI Video & Animation stream, Tyler demonstrates how to create and stylize depth map animations using Comfy UI and Anime Diff. The process involves generating a depth map from a text prompt using a model by Phil (Machine Delusions) and then applying various styles to the map for a unique animation. Tyler guides viewers through two workflows, the first for generating the depth map and the second for stylizing it. The stream is interactive, with audience members submitting prompts for the AI to generate animations. The session also highlights the potential for endless creativity with AI animations and the importance of experimenting with prompts and models to achieve desired results. Tyler also previews an upcoming guest creator stream with Spence, a notable figure in the AI art community.


  • πŸŽ‰ Tyler is excited to showcase creating depth map animations using Comfy UI and Anime Diff.
  • πŸ“Œ The process involves two workflows: one for generating the depth map and another for stylizing it with Anime Diff.
  • πŸ”— Links to download the necessary workflows and the specific LAURA model created by Phil (Machine Delusions) are provided in the chat.
  • πŸ–₯️ The depth map is generated in a resolution of 512 by 896, optimized for vertical content creation.
  • πŸ“Ή The depth maps created are not always perfectly sensible but are suitable for stylization in animations.
  • βš™οΈ The use of Photon LCM and the control LAURA model helps in achieving smoother and faster animations.
  • πŸ“ˆ The batch prompt scheduler allows for creating prompt traveling depth maps, enhancing creativity.
  • 🎨 The IP adapter can be used to apply specific styles onto the depth maps, although it's mostly operated using textual prompts.
  • πŸ’‘ Randomizing the seed when generating depth maps can yield significantly different results, offering more creative options.
  • πŸ“‰ The workflows are designed to be VRAM efficient, making them accessible to users with limited graphics memory.
  • πŸ”„ A color correction node is used to ensure high contrast and no color in the depth maps, focusing on black, white, and gray tones.

Q & A

  • What is the main topic of the video and animation stream presented by Tyler?

    -The main topic of the stream is generating depth map animations in Comfy UI and then stylizing them using Anime Diff.

  • Who created the depth map model used in the workflow?

    -The depth map model was created by Phil, also known as Machine Delusions.

  • What is the purpose of using a depth map in animation?

    -Depth maps are used to add a sense of depth and dimension to 2D animations, which can then be stylized to create more realistic or artistic visuals.

  • How does the batch prompt scheduler function in the workflow?

    -The batch prompt scheduler allows for the creation of a prompt traveling depth map, meaning it can generate a sequence of depth maps based on different prompts.

  • What is the role of the LCM model in the workflow?

    -The LCM (Latent Convolutional Model) is used to create the depth maps more efficiently and quickly.

  • Why is the IP adapter used in the second part of the workflow?

    -The IP adapter is used to apply a specific style or image onto the depth map, allowing for greater customization and creativity in the final animation.

  • What is the significance of the motion scale in the LCM square root linear setting?

    -The motion scale in the LCM square root linear setting affects the smoothness and fluidity of the motion in the generated animations.

  • How does the color correction node contribute to the depth map generation?

    -The color correction node is used to increase the contrast and reduce the saturation to ensure that the depth map is clean, with no color interference.

  • What is the advantage of using the Shatter motion model by Pixel Pusher?

    -The Shatter motion model is favored for its ability to create interesting and dynamic motion effects in the animations.

  • Why is it recommended to randomize the seed when generating depth maps?

    -Randomizing the seed leads to different outcomes in the generated depth maps, allowing for a wider variety of results and increasing the chances of getting a desirable output.

  • What is the purpose of the highres fix upscaler in the workflow?

    -The highres fix upscaler is used to improve the resolution of the final animation, making it suitable for higher quality presentations or larger displays.



πŸŽ‰ Introduction to Depth Map Animations

Tyler, the host, welcomes viewers to a video and animation stream focusing on creating depth map animations. He introduces the plan for the session, which involves generating depth map animations using Comfy UI and Anime Diff, and then stylizing them. The required resources, including a specific Laura model by Phil (Machine Delusions) and two workflows, are shared through provided links. The process is described as accessible and creative, with an emphasis on the potential for stylization using depth maps.


πŸ“ Setting Up Workflows for Depth Map Creation

The host provides an overview of the first workflow, which is designed to generate a depth map from a prompt. He explains the role of the LCM Laura model and the settings used, such as the resolution and the strength of the LCM Laura. The importance of the batch prompt scheduler is highlighted for creating prompt traveling depth maps. Tyler also discusses the use of prompts from the audience to generate various depth map animations and mentions the potential for VRAM usage during the process.


🌟 Generating and Stylizing the Depth Maps

Tyler demonstrates the process of generating a depth map using a specific prompt and the LCM Laura model. He discusses the use of motion scales and the importance of randomizing seeds for different outcomes. The host also addresses the use of an IP adapter to enhance the depth map with a specific image. He shares examples of generated depth maps and explains how they will be used in the second part of the workflow for stylization.


🎭 Animating and Smoothing Out the Depth Map

The host moves on to the second workflow, which focuses on animating and stylizing the depth map. He details the use of the depth control net and the control GIF to smooth out animations. Tyler also mentions the possibility of using an IP adapter for stylization and discusses the VRAM usage during the animation process. The goal is to create animations that can be smoothed out and slowed down through interpolation.


πŸ§™β€β™‚οΈ Creating a Wizard and Clock Depth Map

Tyler attempts to generate a depth map for a wizard and a clock based on prompts from the audience. He discusses the challenges in getting the desired outcome for the clock prompt and explores different models and seeds to achieve better results. The host also touches on the potential of using depth map pre-processors to create depth maps from normal videos.


πŸ–ΌοΈ Skinning Depth Maps with IP Adapter

The host discusses the ease of 'reskinning' depth maps using the IP adapter or by prompting. He emphasizes the potential for creating music visualizers or cool loops by putting time and effort into the process. Tyler also interacts with the audience, taking more prompts for depth map generation and showing how to apply different styles to the generated maps.


πŸ’» Navigating VRAM Constraints and Workflow Links

Tyler addresses the VRAM constraints when running the workflows and provides solutions for users with limited VRAM. He shares links to the required LCM Laura model and the workflows, ensuring that the audience can access them easily. The host also discusses the process of generating a depth map from text and applying styles to create visually appealing animations.


πŸ› Experimenting with Bug-Themed Depth Maps

The host experiments with creating depth maps using bug-themed prompts. He uses a motion model trained on ants to add a crawling effect to the animations. Tyler also discusses the potential of using different motion models to achieve varied animation effects. The audience is encouraged to generate insect-themed images for use in the IP adapter.


🌐 Sharing Workflows and Encouraging Community Interaction

Tyler shares the workflows and models used during the stream and encourages the audience to download and experiment with them. He highlights the importance of community interaction and sharing creations on the workflow page. The host also previews the next stream with a guest creator, generating excitement for future content.


πŸ“š Finalizing the Stream and Preparing for the Next

The host wraps up the stream by summarizing the activities and expressing gratitude to the audience for their participation. He provides instructions for using the workflows and models, and encourages viewers to share their creations on the workflow page. Tyler also teases the next stream, which will feature a guest creator and promises to be insightful and inspiring.



πŸ’‘Depth Maps

Depth maps are grayscale images that represent the distance of each pixel from the viewer. In the context of the video, they are used to create a sense of three-dimensionality for animation. The process involves generating these maps from text prompts using an AI model, which is then used to animate characters or objects in a 3D space.

πŸ’‘Comfy UI

Comfy UI refers to a user interface designed for ease of use, often associated with software or applications that facilitate the creation of digital content. In the video, it is the interface through which the host interacts with the AI model to generate depth maps and animate them.

πŸ’‘Animate Diff

Animate Diff is a tool or process mentioned in the video that is used to stylize animations. It takes the depth map animations and applies a certain style to them, which can greatly enhance the visual appeal and create unique looks for the animations.

πŸ’‘AI Model

An AI model, short for artificial intelligence model, is a system designed to perform tasks that typically require human intelligence, such as understanding natural language or recognizing images. In this video, the AI model is used to generate depth maps from textual descriptions.


A workflow in the context of the video refers to a sequence of steps or processes that are followed to complete a task. The host discusses two workflows: one for generating depth maps and another for stylizing them with Animate Diff.


Text-to-video is a process where textual information is used to generate video content. In the video, the host uses an AI model to convert text prompts into video depth maps, which are then used for animation.


VRAM, or video random-access memory, is the memory used by a computer's graphics processing unit (GPU) to store image data for rendering or output. The script mentions VRAM in the context of the system requirements for running the workflows without issues.


Interpolation is a mathematical technique that estimates unknown values based on known values. In the video, it is used to smooth out animations by effectively doubling the frame rate, making the transitions between frames appear more fluid.

πŸ’‘Prompt Travel

Prompt travel is a creative process where a series of text prompts are used to guide the AI model through a sequence of images or animations. The host plans to use this technique to create a depth map animation with various prompts from the audience.

πŸ’‘IP Adapter

An IP adapter in this context seems to refer to a tool or feature that allows specific images to influence the style or output of the AI model. The host mentions using the IP adapter to 'skin' the depth map animations with particular styles or images.

πŸ’‘LCM (Latent Convolutional Maps)

LCM, or Latent Convolutional Maps, is a term that refers to a method used in AI for generating images from a latent space. In the video, the host uses an LCM model to create depth maps quickly and efficiently.


Tyler introduces the Civitai AI Video & Animation stream and expresses excitement for the upcoming content on depth maps.

The stream focuses on generating depth map animations using Comfy UI and Anime Diff to stylize them.

Tyler mentions the use of a depth map AI model trained by Phil, also known as Machine Delusions, who previously animated the Civitai homepage.

The depth maps created may not follow real-world shading rules but are suitable for animation purposes.

The stream demonstrates the workflow for creating depth maps and emphasizes the endless creative possibilities.

The depth map animations are generated in 1.5 LCM for faster processing.

Tyler invites viewers to submit prompts for the creation of depth map animations.

The stream showcases the use of the Batch Prompt Scheduler for prompt traveling depth maps.

Tyler explains the technical setup for the first workflow, including the use of the depth map AI model and LCM settings.

The second workflow is introduced for stylizing the depth map animations using Anime Diff.

Tyler demonstrates the adjustment of settings to achieve desired effects in the depth maps.

The stream highlights the importance of randomizing seeds for depth map generation.

Tyler discusses the potential of using depth maps for music visualizations and creating loops.

The stream concludes with a demonstration of the final depth map animation, emphasizing the potential for creative exploration.

Tyler encourages viewers to experiment with the provided workflows and share their creations on Civitai.

The stream ends with a teaser for the next session featuring guest creator Sir Spence, who will showcase his work with Comfy UI and audio-reactive visuals.