Runway Research has announced its upcoming Gen-2 generative AI model, which will give creators the ability to convert words into video.

Runway's Gen-2 video generation model is trained on "an internal dataset of 240M images and a custom dataset of 6.4M video clips." In other words, the dataset is huge, but it is unclear if this custom dataset was created by scouring the web (open in new tab) for videographers' work. If you intend to monetize the content you generate, it may not be the best choice to use.

Still, Runway appears to be a fairly powerful evolution of the Gen-1 tool that many are already using for storyboarding and pre-production visuals; similar to Meta's AI video generator (opens in new tab), the Gen-2 model has some really interesting modes that seem to have been added.

Soon, the tool will be able to generate video using only text. Not only that, but there are several other modes that the company is adding to its plethora of video editing tools, some of which work by feeding the algorithm a video clip, which can then be reworked in other styles, etc.

One exciting example in the teaser video (opens in new tab) is a panning shot across several books on a table; the Gen-2 model transforms it into a nighttime city scene with the books turned into skyscrapers, not the most realistic shot but an idea would be a powerful tool for visualization.

The accompanying research paper (opens in new tab) describes the process in more detail. It has come a long way since its conception, and as is evident from the teaser video, the realism factor of the output has been greatly improved.

It will be interesting to see what the results will be and how this will change the industry as these models become more and more sophisticated; as Runway states on its about page, "Deep learning technology applied to audiovisual content will change art, creativity, and design tools forever."

Signing up for Runaway (opens in new tab) presents the option to get a free plan and some paid versions, but there does not appear to be an option to register interest in the Gen-2 model at this time. likely to launch Gen-2 as an open paid beta, but revealed that it is currently a bit overwhelmed with interest.

Gen-2 is also expected to come with the ability to train its own custom AI video generator, as it can with many image models.

If you haven't yet read the article (opens in new tab) in which Dave describes the process of training a Diffusion model on his uncle's art, it can be quite a painstaking process, but a very rewarding one. I hope Runway makes this process a little easier.