Stable Diffusion 3 is HERE! MASSIVE Improvements, Turbo, 3D, Can Stability AI Survive?
TLDRStability AI has announced the release of Stable Diffusion 3 and Stable Diffusion 3 Turbo on their developer platform API, in partnership with Fireworks AI. Despite recent challenges including CEO departure and restructuring, the company has delivered significant improvements in text-image generation, potentially outperforming competitors like Dolly 3 and Mid Journey V6. The new model features a multimodal diffusion transformer architecture, enhancing text understanding and spelling capabilities. However, the release comes with a new licensing model requiring a Stability AI membership for access to model weights, which may impact the community's engagement and the sharing of modifications on platforms like Hugging Face. Pricing for using the API is detailed, with costs ranging from 4 cents for Turbo images to 25 cents for upscaling to 4K. The announcement has sparked discussions on the balance between innovation, cost, and accessibility in the generative AI space.
Takeaways
- π Stable Diffusion 3 and its Turbo version have been released on Stability AI's developer platform API.
- π€ Stability AI has partnered with Fireworks AI for API orchestration, aiming to deliver an enterprise-grade solution with 99.9% service availability.
- π° A new Stability AI membership model has been introduced, which is required to access the model weights for self-hosting.
- π The efficiency of Stable Diffusion 3 is claimed to be roughly 10 times the cost of SDXL when used through the same API.
- π The release includes impressive artwork demonstrations, showcasing the model's capabilities in creating detailed and cohesive scenes from text.
- π There has been a recent corporate restructuring at Stability AI, which had raised concerns about the company's financial viability.
- π¦ The model weights for Stable Diffusion 3 will be made available for self-hosting to Stability AI members in the near future.
- π The new multimodal diffusion Transformer architecture is said to improve text understanding and spelling capabilities compared to previous versions.
- π‘ The pricing for using Stable Diffusion 3 is detailed, with different costs for image generation, upscaling, in-painting, and video generation.
- β There are questions about the future of Stability AI, including whether the new licensing model will affect how the community interacts with and modifies the model.
- π Stability AI's research paper claims that the new model equals or outperforms state-of-the-art text-image generation systems in various evaluations.
Q & A
What has been the recent development with Stability AI that has caused concern?
-Stability AI has faced challenges including the departure of their CEO to work on a crypto project, corporate restructuring, and issues with paying their GPU bills to Amazon and Cori.
What is the significance of the release of Stable Diffusion 3 and Stable Diffusion 3 Turbo?
-The release signifies a massive improvement in Stability AI's capabilities, offering new features and advancements in generative AI, despite recent struggles.
How does Stability AI plan to make the model weights of Stable Diffusion 3 available?
-They aim to make the model weights available for self-hosting to Stability AI members in the near future, which is a new requirement not seen before.
What is the partnership with Fireworks AI about?
-Stability AI has partnered with Fireworks AI to deliver the Stable Diffusion 3 models through their developer platform API, focusing on improving API performance and reliability.
What are the capabilities of the new multimodal diffusion Transformer architecture?
-The new architecture uses separate sets of weights for image and language representations, enhancing text understanding and spelling capabilities compared to older versions of Stable Diffusion.
What is the pricing structure for using Stable Diffusion 3 through the API?
-The cost is roughly 10 times the cost of SDXL when used through the same API, equating to about 7 cents per image generated with Stable Diffusion 3, and around 4 cents per image with Stable Diffusion 3 Turbo.
What is the Stability AI membership and what does it offer?
-The Stability AI membership is a new product offering access to various models hosted online, including image, video, language, and 3D models. It offers different tiers with varying levels of access and commercial usage rights.
How does Stability AI plan to handle the availability of 3D models in their API?
-As of the script's knowledge, there is no current API endpoint for Stable Diffusion 3 that does 3D models, but it is mentioned as part of their offerings, suggesting future availability.
What are the enterprise features offered by Stability AI?
-The enterprise features are not explicitly detailed, but they imply faster GPU response times and potentially more parallelization with job submissions.
How does Stability AI's new licensing model affect the community and model modifications?
-The new licensing model may impact how people fine-tune and post modifications of the models, and could potentially reduce the need for quantizations since the model is more efficient.
What is the community's reaction to the new membership model for accessing Stable Diffusion 3?
-The community's reaction is not detailed in the script, but it is suggested that there may be mixed feelings about the membership requirement for access.
How does Stability AI's partnership with Fireworks AI and their use of Amazon AWS GPUs affect their service reliability?
-The partnership aims to deliver an enterprise-grade API solution with 99.9% service availability, suggesting a push for more reliability and robustness in their service.
Outlines
π Stability AI's New Release and Challenges
Stability AI, a key player in the open-source generative AI field, has faced recent challenges including the departure of their CEO, corporate restructuring, and financial difficulties such as unpaid GPU bills to Amazon and Cori. Despite these issues, they've made a significant announcement regarding the release of Stable Diffusion 3 and Stable Diffusion 3 Turbo on their developer platform API, in partnership with Fireworks AI. This move aims to improve API performance and reliability. The announcement also hints at the potential release of model weights for self-hosting to Stability AI members, which is a new strategy possibly to generate revenue and attract investment. The models demonstrated are capable of creating highly detailed and cohesive scenes from text. However, the release has raised questions about the pricing model and the omission of certain features that were promised earlier.
π‘ Stability AI Membership and Pricing Structure
Stability AI has introduced a new membership model, which offers various tiers of access to their models, similar to a Creative Cloud for AI tools. This membership will provide access to image, video, language, and 3D models hosted online. The membership has different commercial use allowances depending on the tier, with professional membership allowing commercial use. The announcement also introduces Stable Image Core, the API for accessing Stable Diffusion 3. The pricing for using the API is detailed, with costs associated with generating images and other tasks such as upscaling, in-painting, and out-painting. The efficiency and cost of Stable Diffusion 3 are highlighted, with the model being more computationally intensive but offering a more efficient model. The community's reaction to the membership model and the potential impact on model fine-tuning and modifications are topics of interest. The video concludes by inviting viewer engagement and feedback on the pricing and membership model.
Mindmap
Keywords
π‘Stable Diffusion 3
π‘Turbo
π‘Open Source
π‘API
π‘Fireworks AI
π‘Model Weights
π‘Corporate Restructuring
π‘Multimodal Diffusion Transformer
π‘Stability AI Membership
π‘Pricing
π‘Hugging Face
Highlights
Stable Diffusion 3 and Stable Diffusion 3 Turbo have been released on Stability AI's developer platform API.
Stability AI has partnered with Fireworks AI for the fastest and most reliable API platform delivery.
The company has faced recent challenges, including CEO departure and corporate restructuring.
Stability AI aims to make model weights available for self-hosting with a Stability AI membership.
A new licensing model may be introduced, potentially affecting how the community interacts with the models.
The multimodal diffusion Transformer architecture uses separate sets of weights for image and language representations.
Stable Diffusion 3 is claimed to be equal to or outperform state-of-the-art text-image generation systems.
The model is currently only available via API, with an advanced open release in development.
Stability AI membership will be required to access the model weights, possibly as a revenue strategy.
The pricing for using Stable Diffusion 3 is significantly lower than previous versions, at around 7 cents per image.
Stable Diffusion 3 Turbo offers a slightly cheaper rate at approximately 4 cents per image.
Upscaling to 4K with Stable Diffusion 3 costs 25 cents per image.
In-painting and out-painting services are available at 3 and 4 cents per image, respectively.
Video generation with the model costs around 20 cents per video, with durations potentially between 5 to 10 seconds.
Stability AI's new membership model offers commercial and non-commercial access at different pricing tiers.
The company's commitment to open generative AI is reflected in their plans to release models with an AI membership.
Stability AI's potential move away from Hugging Face could indicate a strategic shift in their business model.
The community's reaction to the membership model and its impact on model fine-tuning and modifications remains to be seen.