An AI artist explains his workflow

Vox
2 May 202308:18

TLDRThe AI artist shares his creative process of blending traditional art with Stable Diffusion technology. He begins with a sketch, then uses various prompts to find an initial pose, and refines the artwork in Photoshop. The artist emphasizes the importance of control over AI, using it to enhance his work rather than replace his artistic input. He highlights the challenges of capturing realistic details, such as hands and facial features, and the collaboration between artist and AI in creating unique digital art.

Takeaways

  • 🎨 The artist uses the character Stelfie as a canvas to showcase the potential of Stable Diffusion combined with artist skills.
  • ✍️ The creative process begins with a traditional sketch to capture the desired scene.
  • 🤖 Diffusion models like Stable Diffusion can be cheeky and may lead the artist away from the original idea.
  • 🔍 Random prompts are tried to find a good initial pose, which may require manual adjustments in Photoshop.
  • 🖌️ The artist's use of different samplers in Stable Diffusion affects the realism and details of the artwork.
  • 👤 Stelfie's face is created using a model trained specifically on his features from 3D snapshots.
  • 🛠️ The artist manipulates the AI-generated Muhammad Ali's face to match the real characteristics more closely.
  • 🤸‍♂️ The artist aims for Stelfie to have a more realistic, non-idealized body shape.
  • 💻 A significant portion of the artwork is completed in Photoshop, with lesser amounts in Stable Diffusion and Procreate.
  • 👐 The artist emphasizes the importance of driving the AI, rather than being driven by it, viewing it as an opportunity for new creativity.
  • 🖼️ The final artwork is a collaboration between the artist and AI, with the artist bringing traditional skills into the digital realm.

Q & A

  • Who is Stelfie and what is his significance in the artist's work?

    -Stelfie is a fictional character described as funny and clumsy, who time travels and has incredible adventures. He serves as an alter ego to the artist, despite their physical differences. The artist uses Stelfie to showcase the potential of Stable Diffusion combined with artistic skills.

  • What was the artist's initial goal with the Stable Diffusion project?

    -The artist's initial goal was to capture a scene where Stelfie engages in a boxing match with Muhammad Ali, demonstrating the capabilities of Stable Diffusion alongside good artistry.

  • How does the artist begin the creation process for a new artwork?

    -The artist begins by drawing a sketch, which serves as the foundation for the artwork. This helps to maintain focus on the original idea despite the potential for diffusion models to deviate from it.

  • What role does Photoshop play in the artist's workflow?

    -Photoshop is used to refine and adjust the artwork, particularly when the desired pose or detail is not achieved through Stable Diffusion. The artist may recreate poses or make manual adjustments to features like facial expressions in Photoshop.

  • How does the artist use ControlNet in their work?

    -ControlNet is an extension that, if used today, would significantly reduce the time it takes to reproduce a pose that the artist had created two months prior. It aids in maintaining control over the artistic process.

  • What is the importance of samplers in achieving realism and detail in the artwork?

    -Samplers are crucial for the level of realism and detail in the artwork. Different samplers like Euler and DPM have different effects on the replication of textures like skin, with DPM being particularly effective for realistic skin textures.

  • What do 'steps' refer to in the context of Stable Diffusion?

    -Steps indicate how many times Stable Diffusion works on a prompt. A higher number of steps can lead to more refined results, but it's a balance as too many steps might lead to over-processing.

  • Can you explain the difference between 'inpainting' and 'outpainting' in Stable Diffusion?

    -Inpainting refers to asking the AI to modify specific parts of an image, while outpainting involves asking the AI to imagine and create content outside the existing boundaries of the image based on the context provided.

  • How does the artist handle the creation of Stelfie's face?

    -The artist uses a model trained specifically on Stelfie's face, created by taking snapshots from a 3D model of Stelfie and training the AI with those images. This allows for a more accurate and consistent representation of Stelfie's face in the artwork.

  • What challenges did the artist face when trying to replicate Muhammad Ali's face?

    -Replicating a popular person's face like Muhammad Ali's is challenging due to the need for accuracy and recognition. The artist had to manually adjust features like the nose, jaw, and eyes in Photoshop after using Stable Diffusion to create a base face that resembled Ali.

  • How does the artist achieve a balance between using Stable Diffusion and traditional artistry?

    -The artist achieves balance by using Stable Diffusion for about 50% of the work, with 40% done in Photoshop and 10% in Procreate. They also use their traditional artistry skills to manually adjust and refine the artwork, such as reproducing hands in a more realistic manner.

Outlines

00:00

🎨 Creative Process with Stable Diffusion and Art

The first paragraph introduces Stelfie, a character used to demonstrate the capabilities of Stable Diffusion combined with artistic skills. The creator discusses their approach to starting with a sketch and using Stable Diffusion to generate ideas, emphasizing the importance of maintaining control over the original concept. The goal was to depict Stelfie in a boxing match with Muhammad Ali. Challenges in finding the right pose are mentioned, leading to the use of Photoshop for pose recreation. The role of ControlNet in simplifying the process is highlighted, as well as the significance of samplers in achieving realism and detail. The process involves a mix of Stable Diffusion, Photoshop, and Procreate, with a focus on the training of a model specifically for Stelfie's face using 3D snapshots. The importance of noise strength in achieving good results, especially for faces, is discussed. The paragraph concludes with the creator's intent to depict Stelfie as not super fit and the iterative process of refining the artwork.

05:02

🖌️ Refining Art with AI and Traditional Techniques

The second paragraph delves into the challenges of refining the artwork, particularly in capturing the likeness of Muhammad Ali. The creator discusses the manual adjustments made in Photoshop, such as altering facial features and skin tone, to achieve a realistic representation. The paragraph also touches on the historical context of Ali's physicality compared to modern athletes. The creator emphasizes the importance of the artist's role in the creative process, viewing AI as a collaborative tool rather than a threat. The transition from traditional to digital art is highlighted, with the creator sharing their personal journey and the unique challenges of replicating hands in digital art, which they overcome by using photographs of their own hand.

Mindmap

Keywords

💡Stable Diffusion

Stable Diffusion is a type of AI model used for generating images from textual descriptions. It is capable of creating detailed and diverse visual content, but it can also be unpredictable, leading to sometimes cheeky or unexpected results. In the video, the artist uses Stable Diffusion to capture scenes and generate initial poses for the character Stelfie, showcasing the potential of combining AI with artistic skills to achieve a desired artistic outcome.

💡Stelfie

Stelfie is the artist's alter ego and the main character in the described project. He is depicted as a funny and clumsy individual who time travels and has incredible adventures. The artist uses Stelfie to explore different scenarios, such as a boxing match with Muhammad Ali, and to demonstrate the capabilities of AI in art creation.

💡Photoshop

Photoshop is a widely used digital image editing software that allows users to manipulate and enhance images. In the context of the video, the artist uses Photoshop to refine and adjust the images generated by Stable Diffusion, such as recreating poses, modifying facial features, and improving the realism of the artwork.

💡ControlNet

ControlNet is an extension or tool that aids in the reproduction of specific poses or elements in an artwork. It streamlines the process of recreating a particular artwork or pose by providing a framework or guide for the artist. The artist mentions that with ControlNet, reproducing a pose from a previous artwork would take significantly less time.

💡Samplers

Samplers in the context of AI-generated art refer to different algorithms or methods used to create or refine images. They play a crucial role in determining the level of realism and detail in the final output. For instance, the Euler sampler is described as synthetic and fake, while DPM is noted for its effectiveness in replicating skin textures.

💡Steps

In the context of AI art generation, 'steps' refers to the number of iterations or refinements that the AI model performs based on the user's prompt. A higher number of steps can lead to more detailed and refined images, while a lower number might result in more abstract or less detailed outputs.

💡Inpaint and Outpaint

Inpaint and Outpaint are features in AI image generation that allow artists to modify specific parts of an image or to extend the image beyond its original boundaries. 'Inpaint' is used when the artist wants the AI to alter only certain parts of the image, while 'Outpaint' is employed when the AI is asked to imagine and create content that extends beyond the existing image frame.

💡Procreate

Procreate is a popular digital illustration and art creation software used on mobile devices and tablets. It offers a range of tools and brushes that mimic traditional art techniques, allowing artists to create detailed and diverse artwork. In the video, Procreate is mentioned as one of the tools used by the artist, albeit to a lesser extent compared to Stable Diffusion and Photoshop.

💡Noise Strength

Noise strength is a parameter in AI art generation models like Stable Diffusion that controls the level of variation or 'noise' in the generated images. Adjusting noise strength can influence the overall quality and detail of the output, with higher values potentially leading to more detailed or realistic images, while lower values might produce more stylized or abstract results.

💡Muhammad Ali

Muhammad Ali was a legendary professional boxer and cultural icon, known for his charisma, skill, and impact both inside and outside the boxing ring. In the video, the artist aims to recreate Ali's likeness in the artwork, using Stable Diffusion to generate a face that resembles the famous boxer and then manually adjusting features in Photoshop to achieve a more accurate representation.

💡3D Modeling

3D modeling refers to the process of creating a three-dimensional representation of an object or character using computer graphics software. In the video, the artist mentions creating a 3D model of Stelfie to better understand and replicate the character's face from different angles. This technique is used to train a specific model to generate more accurate facial features for Stelfie.

Highlights

The AI artist uses a character named Stelfie as a representation of himself in his works.

Stelfie is depicted as a clumsy and funny character who time travels and has incredible adventures.

The artist's goal was to showcase the potential of Stable Diffusion combined with good artist skills.

The artist begins with a sketch before using Stable Diffusion and other tools.

Diffusion models can sometimes deviate from the original idea, requiring the artist to steer the process.

The artist uses random prompts to find a good initial pose for the character.

Photoshop is used to recreate poses when a satisfactory one cannot be found with Stable Diffusion.

ControlNet, an extension, significantly reduces the time needed to reproduce poses.

Different samplers are used throughout the process for realism and detail, with DPM being particularly effective for skin replication.

Parameters such as steps, inpaint, and outpaint are crucial for the development of the artwork.

Stable Diffusion is used for 50% of the artwork, with Photoshop and Procreate making up the remaining 40% and 10% respectively.

A specific model trained on Stelfie's face is used, utilizing 3D snapshots for accuracy.

The noise strength setting in Stable Diffusion is important for controlling the final image.

The artist manually adjusts the features of Muhammad Ali's face in Photoshop for accuracy.

The artist aims for Stelfie to appear not super fit, focusing on achieving a realistic pose.

The artist uses Stable Diffusion to refine details like lighting, skin, and muscle definition.

The final artwork is a combination of AI and traditional artistry, with the artist not feeling threatened by the technology.

The artist sees the use of AI as an opportunity for new talent to explore a unique branch of digital art.

The artist's hands are often used in the artwork due to the challenge of reproducing hands with AI.