OpenAI's Sora Made Me Crazy AI Videos—Then the CTO Answered (Most of) My Questions | WSJ

The Wall Street Journal
13 Mar 202410:38

TLDRThe Wall Street Journal explores the capabilities and challenges of Sora, OpenAI's text-to-video AI model, through a conversation with CTO Mira Murati. Sora, a diffusion model, generates one-minute hyper-realistic videos from text prompts. While the technology produces smooth and detailed scenes, it still encounters issues with elements like hands and object continuity. The model learns from a mix of publicly available and licensed data, including content from Shutterstock. OpenAI is focusing on optimizing Sora for wider accessibility, aiming for a cost similar to DALL-E's. The company is also conducting thorough testing, known as red teaming, to ensure the technology is safe and reliable before public release. Concerns about the impact on the video industry and the potential for misinformation are acknowledged, with ongoing research into content provenance and watermarking to distinguish AI-generated content from real videos.


  • 🌟 Sora is OpenAI's text-to-video AI model that generates hyper-realistic, one-minute long videos from text prompts.
  • 🤖 The technology behind Sora is a diffusion model, a type of generative model that creates images from random noise.
  • 🎬 Sora's videos are notable for their smoothness and realism, maintaining continuity between frames for a cinematic effect.
  • 🚧 Despite the high quality, Sora's output still has flaws and glitches, such as issues with hands and color changes in objects.
  • 🛠️ OpenAI is working on ways to edit and improve the generated videos post-production.
  • 🚀 Sora's development includes red teaming to test for safety, security, and reliability, aiming to identify and address vulnerabilities and biases.
  • 🤔 The training data for Sora includes publicly available and licensed content, with specifics remaining somewhat unclear.
  • ⏱️ Video generation with Sora can take several minutes and is more computationally intensive than ChatGPT or DALL-E responses.
  • 💰 Sora is currently more expensive to run than other models like ChatGPT and DALL-E, but OpenAI aims to optimize it for public use.
  • 📅 OpenAI hopes to release Sora to the public, with careful consideration given to its impact on global events like elections.
  • 🖼️ Sora's future policies, similar to DALL-E, may include restrictions on generating content featuring public figures or sensitive content.

Q & A

  • What is Sora and how does it generate videos?

    -Sora is OpenAI's text-to-video AI model. It fundamentally operates as a diffusion model, a type of generative model that creates a more refined image starting from random noise. The AI analyzes text prompts and generates a scene by defining a timeline and adding detail to each frame, resulting in one-minute long, hyper-realistic, and highly-detailed videos.

  • How does Sora ensure the smoothness and realism in its generated videos?

    -Sora achieves smoothness and realism by maintaining continuity between frames, ensuring objects and people appear consistent from one frame to the next. This continuity gives a sense of realism and presence, which is a key feature of Sora's video generation.

  • What are some of the flaws and glitches observed in Sora's generated videos?

    -Despite the smoothness, Sora's videos can show imperfections such as issues with the hands' motion, occasional color changes in objects like cars, and instances where the model does not follow the text prompt closely, leading to unexpected transformations or morphing.

  • Is there a way to edit Sora's generated videos post-production?

    -OpenAI is currently exploring ways to allow users to edit and create with the generated videos. While the ability to fix specific elements like taxi cabs in the background is not immediately available, it is part of the ongoing development to enhance the technology as an editable tool.

  • What kind of data was used to train Sora?

    -Sora was trained using a combination of publicly available and licensed data, which may include content from platforms like YouTube, Facebook, Instagram, and Shutterstock. The specifics of the data used were not detailed in the transcript.

  • How long does it take to generate a video with Sora and what is the computing power required?

    -The generation time can vary from a few minutes depending on the complexity of the prompt. Sora requires significantly more computing power compared to models like ChatGPT or DALL-E, which are optimized for public use. Sora is more expensive to run and is currently a research output.

  • When is Sora expected to be released to the public?

    -Mira Murati, CTO of OpenAI, expressed hope that Sora would be available to the public within the year, but also mentioned that the release could be a few months away, taking into account the need to address issues related to misinformation and harmful bias, especially concerning global elections.

  • What kind of content limitations can we expect with Sora?

    -While specific limitations have not been decided yet, it is anticipated that there will be consistency with other OpenAI platforms, such as DALL-E, where the generation of images of public figures is restricted. OpenAI is in a discovery phase and working with artists and creators to determine the necessary limitations and flexibility of the tool.

  • How is OpenAI ensuring that Sora's generated content is safe and free from harmful biases?

    -Sora is undergoing a red teaming process, which involves testing the tool to ensure its safety, security, and reliability. The goal is to identify vulnerabilities, biases, and other harmful issues. OpenAI is also researching watermarking and content provenance to help distinguish between real and AI-generated videos.

  • What is OpenAI's stance on the potential impact of AI-generated videos on jobs in the video industry?

    -OpenAI views Sora as a tool for extending creativity rather than replacing human jobs. They aim to involve professionals from the film industry and other creators in the development and deployment process to ensure the tool augments human capabilities and addresses economic considerations related to data contribution.

  • How does OpenAI balance the ambition for creating powerful AI tools with concerns about safety and societal impact?

    -OpenAI does not see a conflict between profit and safety guardrails. The real challenge lies in addressing safety and societal questions. While the technology is impressive, OpenAI is focused on finding the right path to integrate AI tools into everyday reality without compromising on safety and ethical considerations.

  • What are the future prospects for AI-generated video technology like Sora?

    -The technology is expected to become faster, better, and more widely available. OpenAI is researching and developing methods to verify the authenticity of content, including watermarking, to address concerns about misinformation. The goal is to confidently deploy these systems once the challenges related to content provenance and trust are resolved.



🎥 Introduction to Sora: OpenAI's Text-to-Video AI

The video introduces Sora, OpenAI's text-to-video AI model, which generates hyper-realistic, one-minute long videos from text prompts. The conversation features Mira Murati, OpenAI's CTO, discussing the technology behind Sora, which is a diffusion model that creates images from random noise. Joanna, the interviewer, expresses both amazement and concern about the technology's potential impact. The video showcases the smoothness and realism of the AI-generated videos, while also pointing out flaws such as issues with hands and color changes in objects. The discussion also touches on the challenges of editing and refining the generated content after the fact.


🚀 Developing and Optimizing Sora for Public Use

The second paragraph delves into the development process of Sora, including the use of publicly available and licensed data for training the AI model. Murati confirms that content from Shutterstock is part of the licensed data used. The video generation process is described as time-consuming and computationally intensive, with the goal of optimizing the technology for low-cost and user-friendly access. The discussion addresses the potential impact on the video industry and the importance of involving creators in the development process. Additionally, there is a focus on safety and ethical considerations, including the red teaming process to identify vulnerabilities and biases, and the decision-making process regarding the types of content that will be prohibited from generation.


🤖 Balancing AI Innovation with Societal Implications

The final paragraph reflects on the broader implications of AI technology, particularly the balance between innovation and societal impact. Murati expresses confidence in the potential of AI tools to extend human creativity and knowledge, despite the challenges of integrating these tools into everyday life. The conversation acknowledges the concerns about misinformation and harmful bias, emphasizing the importance of addressing these issues before widespread deployment. The need for research into content provenance and trustworthiness is highlighted, as well as the ongoing exploration of limitations and policies for content generation. The summary concludes with a recognition of the complexity of balancing profit with safety and societal considerations.




Sora is OpenAI's text-to-video AI model, which generates hyper-realistic, highly-detailed one-minute videos based on a text prompt. It is a diffusion model, a type of generative model that creates a distilled image starting from random noise and learning from analyzed videos. Sora is significant in the video as it represents the cutting-edge of AI technology in video generation, showcasing both its potential and the challenges it faces in terms of accuracy and continuity.

💡Diffusion Model

A diffusion model is a type of generative model used in machine learning to generate data samples similar to a given dataset. In the context of the video, Sora uses this technology to start with random noise and progressively refine it into a coherent video scene. The diffusion model is central to how Sora operates, as it allows the creation of smooth and realistic transitions between frames.

💡Text Prompt

A text prompt is a textual input provided to the AI model to guide the generation of content. In the video, text prompts are used to instruct Sora on what kind of video to create. The prompts define the scene, objects, and actions to be included in the generated video, which is crucial for the AI to produce the intended output.


Continuity in filmmaking refers to the consistent flow of action and narrative from one shot to another. In the context of the video, Sora's ability to maintain continuity between frames contributes to the realism of the generated videos. However, the script also points out instances where Sora's continuity is not perfect, such as with the disappearing yellow cab, indicating areas for improvement.

💡Red Teaming

Red teaming is a process where a tool or system is tested for vulnerabilities, biases, and other potential issues to ensure its safety, security, and reliability. In the video, OpenAI is undergoing red teaming with Sora to identify and address any harmful aspects before making it publicly available. This process is vital for responsible AI development and deployment.

💡Public Figures

Public figures are individuals who are widely known by the public, such as celebrities or political leaders. The video mentions that, similar to DALL-E, Sora may have a policy against generating videos of public figures to prevent misuse and potential legal issues. This reflects the ethical considerations and limitations that OpenAI is contemplating for Sora's capabilities.


Watermarking is the process of embedding a digital signature or mark into a video or image to identify its source or authenticity. In the video, OpenAI is researching watermarking techniques for Sora-generated videos to help distinguish them from real videos. This is important for combating misinformation and ensuring trust in the content's origin.


Misinformation refers to the spread of false or misleading information, which can have serious consequences. The video discusses the potential risks of Sora being used to create misleading videos, and the company's commitment to addressing these issues before releasing the technology to the public. Misinformation is a key concern for AI-generated content, highlighting the need for responsible development.

💡Computing Power

Computing power refers to the ability of a computer system to perform calculations and process data. Sora requires significant computing power to generate its detailed videos, which is currently much higher than that needed for a ChatGPT response or a DALL-E image. The video mentions that OpenAI is working on optimizing Sora to make it more accessible and affordable for public use.

💡Artistic Control

Artistic control pertains to the level of influence an artist or creator has over the final output of a creative work. In the context of the video, OpenAI is collaborating with artists to determine the appropriate level of flexibility and control that Sora should provide. This is important for ensuring that the tool can be used effectively in various creative settings without compromising artistic integrity.

💡Content Provenance

Content provenance involves establishing the origin and authenticity of content, which is particularly relevant for AI-generated videos. The video discusses the challenges of determining whether a video is real or AI-generated, and the importance of research in this area. Content provenance is crucial for maintaining trust in media and preventing the spread of manipulated or false content.


Sora is OpenAI's text-to-video AI model that creates hyper-realistic, highly-detailed one-minute videos from text prompts.

Sora is based on a diffusion model, a type of generative model that starts with random noise to create a distilled image.

The AI model analyzes numerous videos to learn object and action identification, crafting scenes with a defined timeline and detailed frames.

Sora's videos are praised for their smoothness and realism, akin to the continuity and consistency required in traditional filmmaking.

Despite the realism, there are still noticeable flaws and glitches, such as issues with hand motion and color changes in objects.

OpenAI is working on improving Sora's ability to follow prompts more closely and correct imperfections like the disappearing yellow cab.

Sora's development includes red teaming to test for safety, security, reliability, and to identify potential biases and vulnerabilities.

The AI model's training data includes publicly available and licensed content, with confirmed inclusion of Shutterstock videos.

Generating a Sora video can take a few minutes and requires significant computing power, making it more expensive than ChatGPT or DALL-E responses.

OpenAI aims to optimize Sora for public use, targeting a similar cost to DALL-E and a potential release within the year.

The release timeline for Sora is cautious, considering global events like elections, to avoid potential misinformation and harmful bias.

Sora's future policies will likely mirror those of DALL-E, including restrictions on generating images of public figures.

OpenAI is collaborating with artists and creators to determine the level of flexibility and control needed for Sora's tool.

The company is actively researching methods for content provenance, including watermarking, to distinguish AI-generated videos from real ones.

Mira Murati, CTO of OpenAI, emphasizes the importance of addressing safety and societal questions before broadly deploying AI tools.

Sora and similar AI tools are seen as extensions of human creativity, with the potential to greatly enhance our collective imagination and capabilities.

OpenAI is committed to finding the right balance between the advancement of AI and the safety guardrails necessary for responsible deployment.