The battle is heating up in the AI video sector. While the promise of OpenAI's SORA has not yet manifested, there are now some new entrants like KlingAI which are really setting the pace. One of the veterans of the game is RunwayML, the New York based pioneer of AI video which has just released Gen-3 of its already very competent suite of video products.
The increasingly crowded AI video field raises the bar on quality and performance. Where a year ago, janky distorted video clips of actors eating spaghetti were acceptable, we're now seeing some truly spectacular and lifelike moving images reach our screens. For Runway to compete means a dramatic update of its technology, both in terms of resolution as well as coherence and video fidelity.
It's important to note that Runway is not just a video generating tool. In fact the platform is crammed with a huge variety of tools to edit, customize and enhance video and images. As well as text to image and green screen background remover. There are also tools to do image to video, generative audio, upscaling and frame interpolation. A grand total of 32 media tools for just about any media editing purpose you can think of.
There are five plans, ranging from a free plan with 125 credits for 25 watermarked images (which can't be renewed) up to an Unlimited Plan at $95 a month for 2250 Gen 3 image credits and the ability to train a model in a specific style. They're not the most generous plans on the market, especially compared to the increasing number of international rivals, but for most people the Standard Plan at $15 a month for 625 Gen 3 image credits is a good place to start.
The home screen dashboard offers instant access to the tools which most users will need on a daily basis. Text to video, video to video, Lip Sync Video and background removal are likely to be very popular. The menu choices are enhanced by the fact you can tag any option as a favorite, at which point it's available at the top of the screen to make navigation even easier.
The business orientation of the product is confirmed by the prominent position of the Assets and Workspace sections in the sidebar. Here it's possible to organize and share all your creations in folders, and under custom tags to make it simple to build a portfolio of business assets that can be reused quickly and easily. It's one of the features that's often overlooked by newer platforms, but it's very important when you have teams working on enterprise wide material day in, day out.
In addition, the use of Projects, which combine all the best bits of video editing with some AI smarts, offer team users a lot of control over their workflow, which is again important to business users. The basic editing functions revolve around a traditional video timeline motif, so it's definitely not just an AI prompt and you're done. Some serious editing can be done on any clip creations, including green screen, motion tracking and filters. At which point things start to get a little complicated.
This comprehensive editing suite is what sets Runway apart from the competition, but this strength is also a bit of a weakness. To get the best out of the platform, users definitely need to spend time going through the extensive tutorials on Runway Academy, the integrated help system. The combination of sophisticated AI and advanced editing, means you can create just about anything you want or can dream of. But it's not an easy job to learn from scratch.
Many users, especially on the lower plans, will likely see Runway as just another AI video generator, and to be fair, they'll be able to get good value out of using it like that. At this level, you're really only going to be using the prompt box, and doing only some basic editing on the results.
For example I set up a quick and dirty Lip Sync Video in around five minutes, and the results were very impressive. However, I'm not sure that using an avatar tool such as HeyGen alongside an ElevenLabs voice would have been any less impressive - and in a shorter and less cumbersome process.
To some extent I can see that this is deliberate. A quick glance at the platform's user gallery highlights the company's aspiration to be an artistic leader, rather than just a jobbing video editing service. This nod towards Hollywood is evident in almost every video we see on the Runway Studios gallery page. Sure you can make cheesy corporate marketing videos, the demos seem to say, but really we want you to use us to create award winning art. It's a great differentiator, in the same way that Midjourney deliberately appeals to the artistic image creator rather than the marketing hacks of the world.
So with that in mind, can we say that Runway really offers what the general market needs? The answer is an equivocal yes. While none of the various tools on offer is probably best in class, together they offer a perfect mix of AI goodies which anyone can dip into as they need. It didn't take me too long to take a lip synced video clip, push it into the green screen editor and replace the background completely. That's something which would have taken a decent amount of work a few years ago, now it took minutes.
I tried most of the tools on the platform, and they all worked as expected. Slow motion made things look very cool indeed, I used scene detection to split up a video into scenes, I ran a backdrop remix to mixed effect, and restored a black image to color. Everything worked, although some of the image generation was distinctly strange, especially faces. I even created a fruity 3D texture to use as part of my imaginary game assets. But all the while I couldn't help feeling that I was missing the point.
The point being, Runway is clearly much more than the sum of its separate tools. Unlike most other AI tools currently on the market, it is designed to be used by real visual artists, not just enterprise users and consumers looking for a quick video hack on a deadline. Just as it takes time to learn how to get the best out of the platform, so it takes an 'artist's eye' to combine the elements into something beautiful and valuable.
The founders of the company met at a New York art school, and the platform has been used for Hollywood blockbusters, music videos for people like Kanye West and for editing TV shows such as The Late Show. This is not your grandma's AI text-to-video product. Leave aside the recent controversy over training the models on YouTube material, and consider the surreal fact that the company actually holds annual AI Film Festivals in New York and Los Angeles.
It's important to remember that while RunwayML is no Adobe Inc, with a forty year software legacy, the company's Gen X tools have arguably done more to upend the world of video editing than any other company in recent history - and in a mere eighteen months since first product launch. While Avid and Adobe hold top video editing spots among professional film makers, the sheer depth and versatility of Gen 3 has thrown down the gauntlet in terms of delivering high quality editing results using AI smarts.
However, that being said, the infant platform still has a way to go to mature into a true rival for the crown. For one thing, obtaining a top notch result at a professional level is still quite an arduous process. The company's heritage as a co-developer of the Stable Diffusion image generation technology is both a strength and a weakness. It's clear that the focus on delivering innovative video has overshadowed development work on image gen, and it shows in the weak coherence and fidelity from many of the Runway Gen 3 results. But Gen 3 is still in Alpha, so perhaps we should forgive.
But it's the interface that really calls out for help at this stage, and before it's too late. For some reason the user experience is rather like wading through a messy teenager's bedroom. There's stuff strewn everywhere, in no discernible order or pattern. Why for example, are there two separate headline menu options for Generative Audio and Lip Sync Video when they are identical functions?
And why are there no two feature interfaces which are consistent in look and feel? Each component of the platform is confusingly different, even where the functionality and user interface could easily be more consistent. And whose brilliant idea was it to bury the prime video editing function beneath a 'More' button in the sidebar? Ouch!
To be fair to RunwayML, UX and UI are often the very last items to be addressed in the mad dash to achieve product market fit. Building out features, fixing bugs and responding to market aspirations often quite rightly consume all the air in the room, leaving little for the poor customer's usability needs. Just ask eBay. But we're rapidly approaching the end of the honeymoon period with AI applications, and if the company doesn't address some of the main issues, someone else is very likely to come along and steal the market.
My experience of the platform, although constrained by my utter lack of artistic skills, was enough to demonstrate that this is a product which has a shining future for all the right reasons. It's not polished, but it's clearly extremely powerful, and contains all the elements needed to build out a rival to anything else on the market in terms of multimedia creation and editing. The seamless AI integration is definitely a bonus.
At the end of the day, I couldn't help feeling that I was outclassed by the toolkit. My pedestrian efforts at creating an AI video clip was barely scratching the surface of what Runway is capable of. Yes it can do jobbing AI image and video manipulation tasks, rather like a Ferrari can help you pick up groceries from Walmart. But for those professional video creatives who need to break out of the box and really showcase their talent at the highest level, I can see few rivals to the Runway Gen X tools on offer here.
