OpenAI, the company behind the large language model (LLM) artificial intelligence (AI) ChatGPT, has announced Sora, a new text-to-video AI.
Sora can “create realistic and imaginative scenes from text instructions,” according to OpenAI and it’s already got the internet talking.
Reactions online have been highly varied, with some X users being very enthusiastic, whereas some are a lot more wary.
Tech vlogger and reviewer Marques Brownlee, said in a Tweet with some videos from Sora “If this doesn’t concern you at least a little bit, nothing will.”
Gabor Cselle, who was a Director at Google for over six years, prepared a comparison between Sora and several other popular video generation tools, with Sora looking extremely promising.
.@OpenAI SORA vs @pika_labs vs @runwayml vs @StabilityAI Video.
I gave the other models SORA's starting frame. I tried my best prompting and camera motion techniques to get the other models to output something similar to SORA.
SORA's just much better at longer scenes. pic.twitter.com/TK7QR6jaUK
— Gabor Cselle (@gabor) February 16, 2024
Elon Musk, who helped found OpenAI, didn’t specifically comment on the launch of Sora. However, he did share a screenshot of a user criticizing the company’s current CEO Sam Altman for making the formerly not-for-profit open-source company a closed-source profit-driven company. Make of that what you will.
Multiple people have highlighted the potential this tool has fo’r amateur filmmakers.
One user said: “2023 – ‘Hollywood is really overreacting, AI still very far from movie-like quality, look at those hands’ “2024 – ‘introducing Sora, OpenAI’s text-to-video model’ “, while another shared the results of his experimentation with the tool, stating that “People aren’t taking the ‘everyone will be filmmakers’ seriously enough.”
A senior data scientist for Nvidia shared some insight into how he believes the tool might work and how it may have been trained.
If you think OpenAI Sora is a creative toy like DALLE, … think again. Sora is a data-driven physics engine. It is a simulation of many worlds, real or fantastical. The simulator learns intricate rendering, "intuitive" physics, long-horizon reasoning, and semantic grounding, all… pic.twitter.com/pRuiXhUqYR
— Jim Fan (@DrJimFan) February 15, 2024
What are the limitations of Sora?
OpenAI has admitted openly that Sora is not without limitations and there is plenty of room for improvement. There are two key limitations currently:
- Sora might miss out on details based on cause and effect. The example OpenAI gives is that Sora might be able to generate a video of someone taking a bite out of a cookie, but that cookie might not subsequently have a bite out of it.
- Sora might miss out on some finer spatial details like left and right, or specific camera directions.
How can I use Sora and is it free?
Currently, Sora is not available to the general public to use and OpenAI has not made any statement about whether it will be free to use when it does launch, or if there will be a tiered access approach such as with ChatGPT.
Sora is currently in a testing phase for safety and security. Cybersecurity experts are currently hands-on testing the tool. OpenAI is also working on ways to ensure Sora is not used maliciously, such as a detection tool that can tell if a video was generated by Sora. This will help to prevent the tool from being used to spread misinformation.
The tech company will also be engaging directly with policymakers, according to their blog. They acknowledge that while the tool likely has many beneficial effects that they cannot foresee, there are also probably many ways the tool could be abused that they can’t predict.
The race between AI video generation tools is really heating up and users will soon have a huge range of tools at various price points to choose from.
Featured image credit: Screenshot from a video generated by Sora
