Stable diffusion prompt tutorial. NEW PROMPT BOOK released!

Sebastian Kamph
2 Nov 202230:07

TLDRThe video offers an in-depth tutorial on crafting effective prompts for generating images using the Stable Diffusion model. It introduces the OpenArt prompt book, a resource that guides users on creating prompts with various details such as subject, lighting, environment, and point of view. The tutorial emphasizes the importance of the order of words in a prompt, the use of modifiers to alter style or perspective, and the impact of specific camera lenses and artists' styles on the output. It also covers advanced techniques like mixing artist styles and using 'magic words' for higher resolution and detailed images. The host shares practical tips for prompt engineering, such as using specific terms for consistency and leveraging image-to-image variations for better results. The video is an informative guide for both beginners and experienced users looking to enhance their AI-generated artwork.


  • πŸ“š There's a new prompt book released by OpenArts that serves as a manual for creating effective prompts for generating images.
  • πŸ” The process of crafting prompts is referred to as 'prompt engineering', which involves carefully choosing words to guide the AI in creating the desired image.
  • πŸ€” Start by asking questions to determine the type of image, subject, special details, environment, color scheme, and point of view.
  • πŸ–ΌοΈ You can specify art styles, such as 3D render or Studio Ghibli, and even particular camera lenses to influence the style of the generated image.
  • 🌟 The order of words in the prompt matters, as it can affect the weight given to certain aspects of the image, such as whether an object appears in the sky or not.
  • πŸ“Έ Modifiers like photography style, lighting, and environment can significantly change the style, format, or perspective of the generated image.
  • 🎨 Including specific artists in your prompts can help achieve a particular style, but it's important to research their style to get consistent results.
  • 🌈 The use of 'magic words' like 'HDR', 'Ultra HD', and '64k' can influence the resolution and detail of the generated image.
  • πŸ’‘ Studio lighting and cinematic lighting are mentioned as being more consistent in style, which can be useful for achieving a desired look.
  • πŸ”§ Conventional tools like face restoration can be used to fix issues with generated images, such as incorrect facial features.
  • πŸ”„ Iterating with image-to-image variations can help refine the output, getting closer to the desired result with each iteration.
  • πŸ“ˆ The importance of token efficiency is highlighted, as prompts are limited in length, and the order and length of the prompt can influence the generated image.

Q & A

  • What is the purpose of the 'prompt book' mentioned in the transcript?

    -The 'prompt book' is a resource that provides tips and tricks for creating prompts to generate images using AI models like Stable Diffusion. It guides users on how to write prompts effectively to get desired image outputs.

  • What is the significance of the order of text in a prompt according to the transcript?

    -The order of text in a prompt is significant because it can affect how the AI interprets and prioritizes the elements of the prompt. Placing more important aspects earlier in the prompt can give them more weight in the image generation process.

  • How does the transcript suggest one can improve the consistency of lighting in generated images?

    -The transcript suggests that while AI isn't perfect at replicating consistent photography lighting, using specific terms like 'cinematic lighting' or 'butterfly light' can sometimes result in more consistent and desired lighting effects in the generated images.

  • What role do 'modifiers' play in the image generation process as discussed in the transcript?

    -Modifiers are words that can change the style, format, or perspective of the generated image. They can include terms related to photography, art styles, aesthetics, and more, which help guide the AI to produce images with specific visual characteristics.

  • Why is it recommended to specify the environment or context in a prompt?

    -Specifying the environment or context in a prompt helps the AI to generate images that are more aligned with the desired setting, whether it's indoor, outdoor, or a specific type of environment like a studio or a natural landscape.

  • How can including the name of an artist in a prompt influence the style of the generated image?

    -Including the name of an artist in a prompt can lead to images that resemble the style of that artist's work. However, it's recommended to research the artist's style to ensure the prompt will yield consistent and desired results.

  • What is the importance of the 'scale' parameter in prompt engineering?

    -The 'scale' parameter, also referred to as CFG or classifier free guidance, determines how closely the AI adheres to the prompt. A higher scale value means the AI will follow the prompt more closely, while a lower value allows for more creative freedom.

  • What does the transcript suggest about the use of 'magic words' in prompts?

    -The transcript suggests that 'magic words' like 'HDR Ultra HD', '64k', and 'highly detailed' can be used to influence the resolution and level of detail in the generated images, although their effectiveness can vary.

  • How can the 'seed' parameter be used to control the randomness in image generation?

    -The 'seed' parameter controls the starting point of the noise that the AI uses to generate an image. A non-random, static seed will produce the same image given the same prompt, while a randomized seed will result in a fully random image each time.

  • What is the recommended approach for beginners regarding the 'step count' in image generation?

    -For beginners, it is recommended to stick with the default step count, which is often around 50 for most samplers. Higher step counts can lead to better images but also result in longer render times and diminishing returns.

  • How can the transcript help users improve their prompts for image generation?

    -The transcript provides a comprehensive guide on prompt engineering, discussing the importance of word order, the use of modifiers, specifying environment and context, incorporating artist styles, and understanding parameters like scale, step count, and seed. It also emphasizes the importance of prompt efficiency and the use of conventional tools for post-generation edits.



πŸ“š Discovering the OpenArt Prompt Book

The video introduces the OpenArt prompt book, a resource for creating writing prompts. It clarifies that the video is not sponsored and expresses the host's personal interest in the topic. The host guides viewers through the OpenArt website, discussing the tips and tricks it provides for crafting prompts. The importance of asking questions to determine the subject, style, lighting, environment, and point of view for the desired image is emphasized. Examples of how specific details and the order of words in a prompt can influence the AI's output are given, highlighting the concept of 'prompt engineering.'


🎨 Exploring Modifiers and Artistic Styles in Prompts

This paragraph delves into the use of modifiers in prompts to alter the style, format, or perspective of an image. It discusses the impact of different lenses, camera types, and lighting conditions on the final result. The host shares insights on how to be more specific with prompts to achieve desired photography styles and how mentioning artists can influence the outcome. The paragraph also touches on various art mediums and the importance of understanding an artist's style before including them in a prompt.


πŸŒ… Mixing Artist Styles and Understanding the Impact of Emotions

The speaker discusses the creative process of mixing different artist styles to produce unique results. It also highlights the use of emotions in prompts to set the atmosphere of a scene, with examples ranging from positive emotions like 'cozy' to negative emotions such as 'loneliness' and 'regret.' The importance of aesthetic choices in creating a desired mood is also covered, with references to styles like psychedelic lion and Miami 80s Vibe.


πŸ” Advanced Prompt Techniques and Magic Words

This section covers advanced techniques in crafting prompts, such as using 'magic words' like 'HDR,' 'Ultra HD,' and '64k' to enhance the resolution and detail of the generated images. It also discusses the impact of lighting, with examples of 'studio lighting' and 'cinematic lighting.' The host talks about the significance of the art station and the use of 3D rendering terms like 'octane render' in prompts. The paragraph concludes with tips on using seeds and samplers for image generation.


🌟 Optimizing Prompts for Better AI Image Generation

The host provides tips for optimizing prompts, emphasizing the importance of being specific and concise due to the token limit in prompts. They discuss the balance between creativity and guidance in prompt engineering, suggesting a scale of 7 to 15 for beginners. The paragraph also explains the concept of image-to-image variation and how to use strength variations to fine-tune the generated images. It concludes with a mention of the host's Ultimate Guide tutorial for further learning.


πŸ–ΌοΈ Showcase of AI-Generated Art and Final Thoughts

The final paragraph showcases various examples of AI-generated art, highlighting the diversity and creativity achievable through prompt engineering. It includes a range of styles from oil paintings to steampunk and magical themes. The host also discusses the potential need for face restoration in some images and the use of tools like 'code former' for improvements. The video concludes with a reminder that the content is not sponsored and an invitation to check out the host's Ultimate Guide for more in-depth information.



πŸ’‘Stable Diffusion

Stable Diffusion is a term referring to a type of machine learning model used for generating images from textual descriptions. It is a part of the broader field of artificial intelligence known as generative models. In the context of the video, Stable Diffusion is the model that the speaker is discussing, particularly how to effectively use it to create compelling images through the crafting of detailed prompts.

πŸ’‘Prompt Engineering

Prompt engineering is the process of carefully constructing a text prompt to guide an AI model like Stable Diffusion to generate a specific type of image. It involves understanding how the AI interprets different elements of the prompt and arranging them in a way that will produce the desired outcome. The video emphasizes the importance of prompt engineering in achieving the best results from the AI image generation process.


In the context of the video, modifiers are additional words or phrases that can alter the style, format, or perspective of the generated image. They can include terms that specify lighting conditions, artistic styles, or specific visual effects. For example, the use of 'cinematic lighting' as a modifier would instruct the AI to generate an image with dramatic and moody lighting, similar to that seen in movies.


Resolution refers to the dimensions of the generated image, typically measured in pixels. The video mentions '5:12x5' as the default resolution for the Stable Diffusion model, meaning the output image would be 512 pixels wide and 512 pixels tall. Higher resolution images like 4K, 8K, or even 64K are mentioned as options to achieve more detailed images, although they may require more computational resources.


A seed in the context of AI image generation is a random number used to initiate the image creation process. The video explains that by setting a specific seed, one can achieve a consistent starting point for the image generation, allowing for iterative refinements by changing only certain aspects of the prompt while keeping the seed constant.


A sampler in the context of the video refers to the algorithm used by the AI to generate the image from the noise. Different samplers can affect the quality of the image and the time it takes to generate it. The video suggests using specific samplers like DDIM for faster results or Euler Ancestral for situations where fewer artifacts are desired.

πŸ’‘CFG Scale

CFG stands for Classifier Free Guidance and the scale is a parameter that determines how closely the AI adheres to the text prompt. A higher CFG scale means the AI will more strictly follow the prompt, while a lower scale allows for more creative freedom. The video suggests a range of 7 to 15 for a balance between creativity and adherence to the prompt.

πŸ’‘Artist Styles

Artist styles refer to the distinctive visual characteristics of a particular artist's work. In the context of the video, including the names of specific artists or art styles in the prompt can guide the AI to generate images that resemble the work of those artists. This can be a powerful technique to achieve a desired aesthetic or mood in the generated images.


Emotion in the context of the video refers to the feelings or atmosphere that the generated image is intended to convey. The speaker discusses including emotions in the prompt to influence the tone of the image, such as 'scared Panda' or 'sad girl', which can help the AI to create images with a specific emotional impact.


Aesthetics in the video pertains to the visual principles or the sensory aspects that make an image artistically beautiful or appealing. The speaker talks about various aesthetic styles like 'psychedelic', 'Miami 80s Vibe', and 'post-apocalyptic', which can be included in the prompt to guide the AI towards creating images with a particular visual appeal or thematic style.

πŸ’‘Image-to-Image Variation

Image-to-image variation is the process of iteratively refining an AI-generated image by using the previous generation as a starting point for the next. The video explains that this technique can be used to correct imperfections or to evolve the image towards a desired outcome, such as fixing facial features or changing the background.


A new prompt book has been released to help with creating stable diffusion prompts.

The prompt book is a compendium of texts providing tips and tricks for crafting effective prompts.

Prompt engineering involves writing text for prompts, with a focus on specificity and detail.

The order of words in a prompt can significantly influence the AI's interpretation and the resulting image.

Modifiers can change the style, format, or perspective of the generated image.

Photography, artist styles, and specific camera lenses can be specified in prompts to influence the output.

Examples provided in the book include creating images with cinematic lighting and vibrant colors.

The importance of specifying the environment and point of view in a prompt is emphasized.

Different art mediums like chalk, oil painting, and watercolor can be requested in prompts.

Including specific artists in prompts can lead to images in their distinctive styles.

Combining different artists or styles in a single prompt can result in unique and creative images.

The prompt book discusses the use of 'magic words' to enhance image resolution and detail.

Parameters such as resolution, CFG scale, and step counts are explained for fine-tuning prompt results.

The importance of token efficiency is highlighted, as prompts are limited in length.

The use of seeds for generating images allows for consistency or variation in output.

Different samplers are available for image generation, each with its own advantages and processing times.

The book provides guidance on when to use different CFG values and how to ensure prompt specificity.

The potential of image-to-image variation and iterative refinement is explored for achieving desired results.