An AI artist explains his workflow
TLDRThe AI artist shares his creative process of blending traditional art with Stable Diffusion technology. He begins with a sketch, then uses various prompts to find an initial pose, and refines the artwork in Photoshop. The artist emphasizes the importance of control over AI, using it to enhance his work rather than replace his artistic input. He highlights the challenges of capturing realistic details, such as hands and facial features, and the collaboration between artist and AI in creating unique digital art.
Takeaways
- 🎨 The artist uses the character Stelfie as a canvas to showcase the potential of Stable Diffusion combined with artist skills.
- ✍️ The creative process begins with a traditional sketch to capture the desired scene.
- 🤖 Diffusion models like Stable Diffusion can be cheeky and may lead the artist away from the original idea.
- 🔍 Random prompts are tried to find a good initial pose, which may require manual adjustments in Photoshop.
- 🖌️ The artist's use of different samplers in Stable Diffusion affects the realism and details of the artwork.
- 👤 Stelfie's face is created using a model trained specifically on his features from 3D snapshots.
- 🛠️ The artist manipulates the AI-generated Muhammad Ali's face to match the real characteristics more closely.
- 🤸♂️ The artist aims for Stelfie to have a more realistic, non-idealized body shape.
- 💻 A significant portion of the artwork is completed in Photoshop, with lesser amounts in Stable Diffusion and Procreate.
- 👐 The artist emphasizes the importance of driving the AI, rather than being driven by it, viewing it as an opportunity for new creativity.
- 🖼️ The final artwork is a collaboration between the artist and AI, with the artist bringing traditional skills into the digital realm.
Q & A
Who is Stelfie and what is his significance in the artist's work?
-Stelfie is a fictional character described as funny and clumsy, who time travels and has incredible adventures. He serves as an alter ego to the artist, despite their physical differences. The artist uses Stelfie to showcase the potential of Stable Diffusion combined with artistic skills.
What was the artist's initial goal with the Stable Diffusion project?
-The artist's initial goal was to capture a scene where Stelfie engages in a boxing match with Muhammad Ali, demonstrating the capabilities of Stable Diffusion alongside good artistry.
How does the artist begin the creation process for a new artwork?
-The artist begins by drawing a sketch, which serves as the foundation for the artwork. This helps to maintain focus on the original idea despite the potential for diffusion models to deviate from it.
What role does Photoshop play in the artist's workflow?
-Photoshop is used to refine and adjust the artwork, particularly when the desired pose or detail is not achieved through Stable Diffusion. The artist may recreate poses or make manual adjustments to features like facial expressions in Photoshop.
How does the artist use ControlNet in their work?
-ControlNet is an extension that, if used today, would significantly reduce the time it takes to reproduce a pose that the artist had created two months prior. It aids in maintaining control over the artistic process.
What is the importance of samplers in achieving realism and detail in the artwork?
-Samplers are crucial for the level of realism and detail in the artwork. Different samplers like Euler and DPM have different effects on the replication of textures like skin, with DPM being particularly effective for realistic skin textures.
What do 'steps' refer to in the context of Stable Diffusion?
-Steps indicate how many times Stable Diffusion works on a prompt. A higher number of steps can lead to more refined results, but it's a balance as too many steps might lead to over-processing.
Can you explain the difference between 'inpainting' and 'outpainting' in Stable Diffusion?
-Inpainting refers to asking the AI to modify specific parts of an image, while outpainting involves asking the AI to imagine and create content outside the existing boundaries of the image based on the context provided.
How does the artist handle the creation of Stelfie's face?
-The artist uses a model trained specifically on Stelfie's face, created by taking snapshots from a 3D model of Stelfie and training the AI with those images. This allows for a more accurate and consistent representation of Stelfie's face in the artwork.
What challenges did the artist face when trying to replicate Muhammad Ali's face?
-Replicating a popular person's face like Muhammad Ali's is challenging due to the need for accuracy and recognition. The artist had to manually adjust features like the nose, jaw, and eyes in Photoshop after using Stable Diffusion to create a base face that resembled Ali.
How does the artist achieve a balance between using Stable Diffusion and traditional artistry?
-The artist achieves balance by using Stable Diffusion for about 50% of the work, with 40% done in Photoshop and 10% in Procreate. They also use their traditional artistry skills to manually adjust and refine the artwork, such as reproducing hands in a more realistic manner.
Outlines
🎨 Creative Process with Stable Diffusion and Art
The first paragraph introduces Stelfie, a character used to demonstrate the capabilities of Stable Diffusion combined with artistic skills. The creator discusses their approach to starting with a sketch and using Stable Diffusion to generate ideas, emphasizing the importance of maintaining control over the original concept. The goal was to depict Stelfie in a boxing match with Muhammad Ali. Challenges in finding the right pose are mentioned, leading to the use of Photoshop for pose recreation. The role of ControlNet in simplifying the process is highlighted, as well as the significance of samplers in achieving realism and detail. The process involves a mix of Stable Diffusion, Photoshop, and Procreate, with a focus on the training of a model specifically for Stelfie's face using 3D snapshots. The importance of noise strength in achieving good results, especially for faces, is discussed. The paragraph concludes with the creator's intent to depict Stelfie as not super fit and the iterative process of refining the artwork.
🖌️ Refining Art with AI and Traditional Techniques
The second paragraph delves into the challenges of refining the artwork, particularly in capturing the likeness of Muhammad Ali. The creator discusses the manual adjustments made in Photoshop, such as altering facial features and skin tone, to achieve a realistic representation. The paragraph also touches on the historical context of Ali's physicality compared to modern athletes. The creator emphasizes the importance of the artist's role in the creative process, viewing AI as a collaborative tool rather than a threat. The transition from traditional to digital art is highlighted, with the creator sharing their personal journey and the unique challenges of replicating hands in digital art, which they overcome by using photographs of their own hand.
Mindmap
Keywords
💡Stable Diffusion
💡Stelfie
💡Photoshop
💡ControlNet
💡Samplers
💡Steps
💡Inpaint and Outpaint
💡Procreate
💡Noise Strength
💡Muhammad Ali
💡3D Modeling
Highlights
The AI artist uses a character named Stelfie as a representation of himself in his works.
Stelfie is depicted as a clumsy and funny character who time travels and has incredible adventures.
The artist's goal was to showcase the potential of Stable Diffusion combined with good artist skills.
The artist begins with a sketch before using Stable Diffusion and other tools.
Diffusion models can sometimes deviate from the original idea, requiring the artist to steer the process.
The artist uses random prompts to find a good initial pose for the character.
Photoshop is used to recreate poses when a satisfactory one cannot be found with Stable Diffusion.
ControlNet, an extension, significantly reduces the time needed to reproduce poses.
Different samplers are used throughout the process for realism and detail, with DPM being particularly effective for skin replication.
Parameters such as steps, inpaint, and outpaint are crucial for the development of the artwork.
Stable Diffusion is used for 50% of the artwork, with Photoshop and Procreate making up the remaining 40% and 10% respectively.
A specific model trained on Stelfie's face is used, utilizing 3D snapshots for accuracy.
The noise strength setting in Stable Diffusion is important for controlling the final image.
The artist manually adjusts the features of Muhammad Ali's face in Photoshop for accuracy.
The artist aims for Stelfie to appear not super fit, focusing on achieving a realistic pose.
The artist uses Stable Diffusion to refine details like lighting, skin, and muscle definition.
The final artwork is a combination of AI and traditional artistry, with the artist not feeling threatened by the technology.
The artist sees the use of AI as an opportunity for new talent to explore a unique branch of digital art.
The artist's hands are often used in the artwork due to the challenge of reproducing hands with AI.