Runway AI Unveils API for Advanced Video Generation, Revolutionizing Content Creation

Runway AI launches an API to expand access to its most powerful video generation model - SiliconANGLE

Runway AI launches an API to expand access to its most powerful video generation model Generative artificial intelligence startup Runway AI Inc. said today that it's making its most advanced video generation model available to companies via an application programming interface that's available in early access now. With the move, organizations will be able to integrate Runway's Gen-3 Alpha Turbo model directly into their own applications, platforms and services, making it easier for their developers and other employees to create new video content with the tools they use for everyday work. New York-based Runway said the new API basically makes its video creation model more accessible, so that advertising teams can create marketing videos on the fly within their existing workflows. The API isn't available to everyone yet, but interested companies can sign up to the waitlist to gain access. The company said in a blog post it wants to obtain feedback from early adopters of the API, before rolling it out to everyone in the coming weeks. The generative AI startup is one of the leading players in video creation, having been founded in 2018 and released a series of increasingly powerful AI models designed for that purpose. Its foundational models power a series of tools that are meant to simplify the process of creating videos, offering capabilities such as real-time video editing, automated rotoscoping and motion-tracking. Those tools, which are aimed at both professionals and hobbyists, are said to dramatically reduce the time and effort required to generate high-quality videos. For instance, its automated rotoscoping tool enables users to quickly separate foreground elements from the background, a task that has traditionally required tons of effort and knowledge of sophisticated editing software. Meanwhile, its motion tracking tool makes it easier to track and apply the effects of moving objects or people in videos. Users can perform both tasks simply by telling it what to do with text-based prompts. Runway announced its Gen-3 Alpha Turbo model back in June, saying at the time that it's the most advanced model it has released so far. It enables users to generate higher-fidelity videos than its previous models could do, with better depiction of motion. The new API will be offered via two different subscription plans, one for individuals and small teams, and another for enterprises. Depending on which plan customers choose, they'll gain access to endpoints that allow them to integrate the latest model with various tools they're already using, so users can initiate video generation tasks without having to disrupt their workflows. Runway said it will charge one cent per credit to access the API, with five credits required to create a one second video. Using the Gen-3 Alpha Turbo model, it's possible to create videos of up to 10 seconds at the most, a task that would cost 50 cents. With the launch of the API, Runway is making the Gen-3 Alpha Turbo model more accessible, and will hope that this leads to wider adoption. Previously, the only way to access it was through the Runway platform. According to the company, the marketing firm Omnicom Group Inc. is already experimenting with the API, though it didn't say what kinds of videos it is producing with it. The debut of the API moves Runway further ahead of its rivals in the generative AI video industry. Competitors such as OpenAI and Deepmind have yet to launch their alternative video generation models. The company has been in the news a lot lately. Days earlier, Runway announced a new capability within its platform called "Video to Video", which is said to represent another way users can introduce more precise movement and expressiveness into their generated videos. They simply upload an existing video that they want to replicate, prompt their desired aesthetic direction, and the model will do the rest. In July, it was revealed that Runway was holding talks with potential investors about the possibility of raising an additional $450 million in funding at a valuation of $4 billion, with plans to use the capital to accelerate the development of its models and expand its developer, sales and go-to-market teams. That month, the company also found itself at the center of some controversy when a leaked document emerged, showing that it had scraped content from "thousands of videos from popular YouTube creators and brands, as well as pirated films", in order to train its video generation models."

Runway debuts API allowing enterprises to build apps, products atop its realistic video AI model

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More As enterprises continue to increase their investments in generative AI, Runway is going all in to give them the best it has on offer. Today, the New York-based AI startup announced that it is making its ultra-fast video generation model, Gen-3 Apha Turbo, available via API. The move makes Runway among the first companies to allow developers and organizations to integrate a proprietary AI video generation model into their platforms, apps, and services -- powering internal or external use cases requiring video content. Imagine an advertising company being able to generate video assets for campaigns on the fly. The launch promises to significantly enhance the workflows of video-focused enterprises. However, Runway noted that the API will not be immediately available to everyone. Instead, the company is following a phased approach, gradually rolling it out to all interested parties. What do we know about the Runway API? Currently available to select partners, the Runway API comes via two main plans: Base for individuals and small teams, and Enterprise for larger organizations. Depending on the plan chosen, users will receive endpoints to integrate the model into their respective products and initiate various video generation tasks, with the interface clearly displaying "powered by Runway" messaging. The base price for the API starts at one cent per credit, with five credits required to generate a one-second video. It's important to note that, at this stage, the company is only providing access to the Gen-3 Alpha Turbo model via the API. Other models, including the original Gen-3 Alpha, are not yet available on the platform. The Turbo model debuted in late July as an accelerated version of Gen-3 Alpha, capable of producing videos from images seven times faster while being more affordable. Runway co-founder and CEO Cristóbal noted at the time that the model could generate videos almost in "real-time," producing a 10-second clip in just 11 seconds. Until now, the model was only available to users on the Runway platform. With the API, the company hopes to see broader adoption across various enterprise use cases, which could ultimately boost its revenues. Runway said in a blog post that marketing group Omnicom is already using the API, although it did not say how exactly the group is putting the video generation technology to use. The names of other existing partners have also not been revealed. Either way, with this announcement, the messaging is pretty clear: Runway is taking a proactive step to stay ahead of competition in the market, including the likes of OpenAI's yet-to-launch Sora and Deepmind's Veo, and gain a bunch of enterprise customers. Not to mention, despite all the criticism surrounding AI video generation, right from copyright cases to questions about data collection for training, the company has been aggressively moving to expand its product capabilities. Just a couple of days ago, it launched Gen-3 Alpha Video to Video on the web for all paid subscribers. "Video to Video represents a new control mechanism for precise movement, expressiveness and intent within generations. To use Video to Video, simply upload your input video, prompt in any aesthetic direction you like, or, choose from a collection of preset styles," the company wrote in a post on X. While it remains to be seen when Runway will add its other models, including Gen-3 Alpha, to the API platform, interested parties can already sign up on the company's waitlist to get access. Runway says it is currently gathering feedback from early partners to further refine the offering but plans to initiate a wider release in the coming weeks to open up access for all waitlisted customers.

Runway announces an API for its video-generating models | TechCrunch

Runway, one of several AI startups developing video-generating tech, today announced an API to allow devs and orgs to build the company's generative AI models into third-party apps, platforms and services. Currently in limited access (there's a waitlist), the Runway API only offers a single model at present -- Gen-3 Alpha Turbo, a faster but less capable version of Runway's current flagship model, Gen-3 Alpha -- and two plans, Enterprise and Build (aimed at individuals and teams). Base pricing is one cent per credit (1 second of video costs 5 credits), and Runway says that "trusted strategic partners" including marketing group Omnicom are already using the API. The Runway API also comes with unusual disclosure requirements. Any interfaces using the API must "prominently display" a "Powered by Runway" banner linking to Runway's website, the company writes in a blog post. "This helps users understand the technology behind your application while adhering to our usage terms," the post continues. Runway, which is backed by investors including Salesforce, Google and Nvidia and was last valued at $1.5 billion, faces stiff competition in the video generation space, including from OpenAI, Google and Adobe. OpenAI is expected to release its video generation model, Sora, in some form early this fall, while startups like Luma Labs continue to refine their technologies. With the preliminary launch of the Runway API, Runway becomes one of the first AI vendors to offer a video generation model through an API. But while the API might help the company along the road to profitability (or at least recouping the high costs of training and running models), it won't resolve the lingering legal questions around those models and generative AI technology more broadly. Runway's video-generating models, like all video-generating models, were trained on a vast number of examples of videos to "learn" the patterns in these videos to generate new footage. Where did the training data come from? Runway refuses to say, like many vendors these days -- partly out of fear of losing competitive advantage. But training details are also a potential source of IP-related lawsuits if Runway trained on copyrighted data without permission. There's evidence that it did, in fact -- a report from 404 Media in July exposed an internal spreadsheet of training data that included links to YouTube channels belonging to Netflix, Rockstar Games, Disney and creators like Linus Tech Tips and MKBHD. It's unclear whether Runway ended up sourcing any of the videos in the spreadsheet to train its models. In an interview with TechCrunch in June, Runway co-founder Anastasis Germanidis would only say the company uses "curated, internal datasets" for model training. But if it did, it wouldn't be the only AI vendor playing fast and loose with copyright rules. Earlier this year, OpenAI CTO Mira Murati didn't outright deny that Sora was trained on YouTube content. And Nvidia reportedly used YouTube videos to train an internal video-generating model called Cosmos. Generative AI vendors believe that the doctrine known as fair use provides them a legal shield. Others aren't taking chances; to train its video-generating models, Adobe is said to be offering artists payments in exchange for clips. If we're lucky, cases making their way through the courts will bring clarity soon enough. However it shakes out, one thing's becoming clear: Generative AI video tools threaten to upend the film and TV industry as we know it. A 2024 study commissioned by the Animation Guild, a union representing Hollywood animators and cartoonists, found that 75% of film production companies that have adopted AI have reduced, consolidated or eliminated jobs after incorporating the tech. The study also estimates that by 2026, more than 100,000 of U.S. entertainment jobs will be disrupted by generative AI.

Runway's latest updates are producing mind-blowing results | Digital Trends

People are also having a field day with Runway's video-to-video generation, which was released on September 13. Essentially, the feature allows you to radically transform the visual style of a given video clip using text prompts. Check out the video below for a mind-altering example of what's possible. Recommended Videos Runway Gen-3 Alpha just leveled up with Video-to-Video Now you can transform any video's style using just text prompts at amazing quality. 10 wild examples of what's possible:pic.twitter.com/onh12zCzpI — Min Choi (@minchoi) September 15, 2024 AI enthusiasts are also producing stunning visual effects that can be displayed on Apple's Vision Pro headset, giving us a potential hint at what developers leveraging the recently announced API will be able to accomplish. X (formerly Twitter) user Cristóbal Valenzuela posted a brief clip to the social media site on Monday showing off the combined capabilities of Gen-3 and Apple Vision Pro. Early experiments rendering Gen-3 on top of the Apple Vision Pro, made by @Nymarius_ pic.twitter.com/SiUNR0vX0G — Cristóbal Valenzuela (@c_valenzuelab) September 15, 2024 The video depicts an open-plan office space with a generated overlay that makes the room appear to be deep jungle ruins. Some users remained unconvinced of the video's veracity, but according to the post, it was generated by someone who actually works at Runway. Twitter user and content creator Cosmo Scharf showed off similar effects in their post, as well as provided additional visual evidence to back up their claims. Runway announced Monday the release of a new API that will enable developers to add video generation capabilities to a variety of devices and apps, though there reportedly are a few restrictions on who can actually access the API. For one, it's only in limited release for the moment, but you can sign up for a waitlist here. You'll also need to be either a Build or Enterprise plan subscriber. Once you are granted access, you'll only be able to leverage the Gen-3 Alpha Turbo model iteration, which is a bit less capable than the company's flagship Gen-3 Alpha. The company plans to charge a penny per generation credit to use the service. For context, a single second of video generation costs five credits so, basically, developers will be paying 5 cents per second of video. Devs will also be required to "prominently display" a "Powered by Runway" banner that links back to the company's website in any interface that calls on the API. While the commercial video generation space grows increasingly crowded -- with Adobe's Firefly, Meta's upcoming Sora, Canva's AI video generator, Kuaishou Technology's Kling, and Video-01 by Minimax, to name but a handful -- Runway is setting itself apart by being one of the first to offer its models as an API. Whether that will be enough to recoup the company's exorbitant training costs and lead it to profitability remains to be seen.

Runway launches new video-to-video AI tool -- here's what it can do

Leading AI video platform RunwayML has finally unveiled its video-to-video tool, allowing you to take a 'real world' video and adapt it using artificial intelligence. Runway launched Gen-3 Alpha, the latest version of its video model in June and has gradually added new features to an already impressive platform that we gave 4 stars and named one of the best AI video generators. It started with text-to-video, added image-to-video soon after, and now it has added the ability to start with a video. There was no video-to-video with Gen-2 so this is a significant upgrade for people wanting to customize a real video using AI. The company says the new version is available on the web interface for anyone on a paid plan and includes the ability to steer the generation with a text prompt in addition to the video upload. I put it to the test with a handful of example videos and my favorite was a short clip of my son running around outside. With video-to-video, I was able to transport him from the real world to an underwater kingdom and then on to a purple-hued alien world -- in minutes. Starting an AI video prompt with a video is almost like flipping the script compared to starting with an image. It lets you determine the motion and then use AI for design and aesthetics. When you start with an image you're defining the aesthetic then AI sets the motion. Runway wrote on X: "Video to Video represents a new control mechanism for precise movement, expressiveness and intent within generations. To use Video to Video, simply upload your input video, prompt in any aesthetic direction you like." As well as being able to define your own prompt there are a selection of preset styles. One can turn the subject matter into a glass effect and another makes it a line drawing. In its demo video we see a sweeping drone view of hills turn first into wool, then into an ocean view and finally sand dunes or clay. Another example shows a city where we first have it at night, then daytime, then in a thunderstorm and finally into bright colors. Being able to film real footage and then use AI to apply either a new aesthetic or even just specific effects -- one example sets off an explosion in the background -- is a significant step forward for generative AI video and adds a new usefulness to the feature.

This AI can turn your mundane video into a special effects spectacular

Your Walk in the Park film becomes a living statue in Rome or an ape in a forest in minutres Runway's AI video creation and editing service has added the promised video-to-video revamp feature to its Gen-3 Alpha model platform. The video-to-video tool lets you submit a video and a text prompt to alter it. The changes can adjust the setting, performers, or other elements of the video to match the text prompt or any of a handful of preset style suggestions. Video-to-video is the last major addition to Runway's video creation options. Runway already allows users to start with either text or images to define the look and motion of the video. Now, by starting with a real-world video, users can define the motion upfront and then use AI to alter the design and aesthetics. Runway trained the feature on a large set of videos and images captioned with details to teach the AI model how to understand uploaded films and the prompts for changing them. It's like seeing a new video from another universe entirely. You can see how it works below. "Video to Video represents a new control mechanism for precise movement, expressiveness, and intent within generations," Runway explained on social media. "To use Video to Video, simply upload your input video, prompt in any aesthetic direction you like, or, choose from a collection of preset styles." When combined with Gen-3 Alpha's existing ability to handle complex transitions and expressive human faces and emotions, the results can be pretty spectacular. Runway has been quick to augment Gen-3 Alpha with new options. Most recently, the company showed off the Gen-3 Alpha Turbo version of its model, which sacrifices some functionality for speed. Along with the video-to-video makeover option, the tool also works with Motion Brush, Advanced Camera Controls, Director Mode, and other, more granular video editing features. Gen-3 Alpha comes off as far more versatile and user-friendly than the Gen-2 model. That said, it does have limitations despite its impressive output. Currently, it can generate video clips up to 10 seconds long, hardly a feature length film. Speed and quality may make up for that until it can go longer. There's ever more competition for Runway to have to outdo, though. The best-known is OpenAI's Sora model, but it's far from the only one. Stability AI, Pika, Luma Labs' Dream Machine, and more are all racing to bring AI video models to the public. Even TikTok's parent company, Bytedance, has an AI video maker called Jimeng. If you want to try out Runway's video-to-video tool, Gen-3 Alpha is accessible to users on paid plans starting at $12 per month.

AI video rivalry intensifies as Luma announces Dream Machine API hours after Runway

The increasingly competitive AI video technology race took another turn on Monday as Luma AI, a San Francisco-based startup founded by former Google, Meta, Adobe and Apple engineers, announced an application programming interface (API) for its Dream Machine video generation model just hours after rival AI video startup Runway announced its own API. The Dream Machine API allows users -- whether they be individual software developers, startup founders, or engineers on teams at larger enterprises -- to build applications and services atop Luma's hit video generation model. As such, it should bring the AI video technology to more apps, teams, and users around the world, and will enable a whole new class of AI video generation features outside of the Luma AI website. Prior to the API launch, the only way to make AI-generated videos with Dream Machine was through Luma's website. AI video models such as Dream Machine and Runway work by training on millions of clips of previously posted footage -- in some cases, without express permission or compensation -- and transforming them into mathematical structures called "embeddings" that can then produce similar or conceptually related visuals based on a user's text prompts or still images that they upload (and which the model automatically converts into motion). Also, unlike rival New York City-based Runway -- which debuted two versions of its API for smaller teams and large enterprises, respectively, both via Google Forms waitlists -- Dream Machine's API is available to begin using now. Already, developers at the New York City-based AI code repository Hugging Face have implemented a demo version on the public Hugging Face website: Amit Jain, co-founder and CEO of Luma AI, explained the company's vision in a statement published as part of a press release, saying: "Our creative intelligence is now available to developers and builders around the world. Through Luma's research and engineering, we aim to bring about the age of abundance in visual exploration and creation so more ideas can be tried, better narratives can be built, and diverse stories can be told by those who never could before." Luma's Dream Machine API and Runway's API both arrived just one weekend after Adobe previewed its "enterprise-safe" Firefly Video AI model -- trained only on data that is public domain or that Adobe has direct license to. But Adobe's Firefly Video is only available to individual users through a waitlist for now, not through an API for enterprises and teams to build separate apps on. Dream Machine's fast rise Dream Machine debuted back in June 2024 as a public beta, instantly wowing users and AI creators with its high degree of realism, relatively fast generation times, and accessibility -- especially in the face of OpenAI's still private Sora model. Luma also previously released a still image, 3D asset generation AI model called Genie via its Discord server. It recently upgraded Dream Machine to add more control via a dropdown of selected camera motions. Now it claims that Dream Machine is the "the world's most popular video model," though VentureBeat is awaiting clarification as to what metric the company is basing this claim upon. Luma Dream Machine API features and capabilities The Dream Machine API is powered by the latest version of Dream Machine (v1.6) and offers several advanced video generation tools: * Text-to-Video: Users can generate videos by simply providing text instructions, eliminating the need for prompt engineering. * Image-to-Video: Static images can be instantly transformed into high-quality animations using natural language commands. * Keyframe Control: Developers can guide video creation with start and end keyframes, controlling the narrative flow. * Video Extension and Looping: The API enables users to extend video sequences or create seamless loops, ideal for UI visuals or marketing content. * Camera Motion Control: This feature lets users direct video scenes through simple text inputs, offering granular control over the generated video's perspective and movement. * Variable Aspect Ratios: The API can produce videos optimized for different platforms, removing the complexity of video and image editing. The Dream Machine API is designed to simplify the process of video creation. Developers can integrate these features into their applications without the need for complex video editing tools, allowing users to stay focused on storytelling and creation. Accessibility and Ppricing One of Luma AI's core goals with the Dream Machine API is to democratize access to high-quality video creation. Jain highlighted the company's dedication to making this technology widely available, stating, "We believe in making these powerful technologies accessible to as many people as possible. This is what we did with the Dream Machine launch, and we have learned an immense amount. I'm excited to learn alongside developers and see what they build with Dream Machine." The API is priced competitively, at $0.32 per million pixels generated, translating to $0.35 for a 5-second video at 720p resolution with 24 frames per second. This pricing model ensures that even smaller developers can experiment with and leverage the platform without facing prohibitive costs. However, without publicly posted pricing by Runway, it is not currently possible to compare the two in terms of value at this time. Scalable for enterprises While the Dream Machine API is open to all developers, Luma AI has also introduced a "Scale" option to cater to larger companies and organizations. This option provides higher rate limits and personalized onboarding and engineering support. According to Jain, the Scale option is a direct response to demand from enterprise clients: "Since day one of Dream Machine, we have had an immense interest from larger companies and organizations asking us about access to our models. So today, we are excited to bring up our Scale option to serve customers and their far-reaching use cases." Responsible use and moderation Luma AI says it uses a multi-layered moderation system, combining AI filters with human oversight to ensure its tech is used responsibly and complies with legal standards. Developers using the API can tailor moderation settings to suit their specific markets and user bases. Luma AI also takes steps to protect user privacy and ownership. Inputs and outputs generated through the API are not used to train Luma's AI models unless explicit permission is granted by the user, ensuring that intellectual property rights remain intact. However, Luma and all other AI video generation model providers have been critiqued by human artists and activists who believe that the tech -- which was presumably trained on videos from around the web, in some cases (perhaps many) without permission or compensation to the owners -- is inherently exploitative and may even violate copyright. Nonetheless, the AI video providers remain undaunted for now. And with the launch of the Dream Machine API, Luma AI aims to further fuel AI video creation around the web, empowering developers to build innovative video tools with ease -- and users to gain further access to tools for expressing their imaginations.

Awesome AI video-to-video restyling effects using Runway ML Gen 3

Ever wondered how you could transform your ordinary video footage into extraordinary, stylized animations without spending countless hours on editing? Enter Runway ML Gen 3. This latest update not only allows you to upload videos but also ensures consistent, flicker-free outputs, making your creative process smoother and more enjoyable. Users can effortlessly upload their video files and apply text prompts to create visually stunning and unique stylized animations. This update ensures improved consistency and minimizes flicker, resulting in smoother and more professional-looking animations that captivate audiences. Check out the video below by AI Animation who takes you through a variety of different styles and techniques you can use on your video. Runway ML Gen 3 is a groundbreaking tool that empowers users to transform their videos into captivating stylized animations using simple text prompts. This latest iteration of the software brings significant enhancements to its previous features, allowing seamless video uploads and generating consistent, high-quality outputs that push the boundaries of video-to-video transformation. Runway ML Gen 3 sets itself apart as a formidable competitor to other tools in the market, such as DomoAI. With its substantial enhancements over previous versions, Runway ML Gen 3 offers users a comprehensive suite of robust and versatile tools for video transformation. These advancements position Runway ML Gen 3 as a top choice for individuals seeking advanced animation capabilities without compromising on quality or ease of use. One of the key strengths of Runway ML Gen 3 lies in its ability to produce a wide array of stylized outputs. Users can effortlessly transform their videos into mesmerizing yarn-inspired animations, lifelike 3D character transformations, and intricate line art illustrations. The tool's versatility extends to supporting imaginative prompts, such as creating a "man made of spaghetti," showcasing its potential to bring even the most unconventional ideas to life. Here are a selection of other articles from our extensive library of content you may find of interest on the subject of video creation and manipulation using artificial intelligence : Runway ML Gen 3 prioritizes a user-friendly experience, offering a straightforward process for uploading videos and applying text prompts. Users can easily customize scene descriptions to tailor the outputs to their specific requirements, ensuring that the generated animations align perfectly with their creative vision. The tool consistently delivers stylized results, making it accessible to both novice users exploring the world of animation and experienced professionals seeking to streamline their workflow. To further support users in their creative journey, Runway ML Gen 3 offers an upcoming AI animation course, providing valuable educational resources to enhance skills and unlock new possibilities. Early access promotions are available, encouraging users to dive in and explore the tool's full potential. The integration of paid tools adds an extra layer of sophistication, empowering professionals with advanced features to take their animations to the next level. Runway ML Gen 3 is extremely powerful and is transform the process of video-to-video animation, offering a powerful and intuitive tool that combines innovative technology with user-centric design. Whether you're a beginner embarking on your animation journey or a seasoned professional seeking to push the boundaries of your craft, Runway ML Gen 3 provides the capabilities and flexibility to bring your creative visions to life with unprecedented ease and quality. As the demand for captivating visual content continues to grow across various industries, from entertainment to marketing, Runway ML Gen 3 emerges as a fantastic option, empowering users to create stunning animations that engage, inspire, and leave a lasting impact on audiences worldwide. With its innovative features, seamless integration, and commitment to user empowerment, Runway ML Gen 3 sets the stage for a new era of video-to-video transformation, unlocking endless possibilities for creators and redefining the future of animation.