Creating video content presents a significant hurdle for generative AI models. Lacking a grasp of spatial awareness or physical principles, these systems produce visuals frame by frame, often leading to noticeable errors. A prior instance highlighted the limitations of AI in this arena, as detailed in a report regarding OpenAI’s Sora, which showcased a video featuring a taxi that inexplicably vanished.

Addressing these specific challenges, AI video firm Runway has reportedly made strides with its latest Gen-4 models. According to Runway, this iteration represents a leap toward “a new generation of consistent and controllable media,” where characters, items, and entire scenes exhibit much more uniformity throughout projects.

For those who have dabbled in AI video production, it’s clear that many videos are short and feature sluggish movement. Elements that exit the frame and re-enter often yield unexpected results, as AI rendering can produce varying representations. Observations such as people merging with buildings or limbs morphing into different objects are frequent occurrences.

This inconsistency arises from the nature of AI functioning as probability engines. They can predict, with some accuracy, how a futuristic urban landscape should appear based on vast datasets but lack comprehension of real-world constructs and fail to maintain a stable memory of environments. Consequently, they tend to re-envision scenes continuously.

Runway seeks to enhance this process by utilizing reference images, which can serve as a consistent point of reference while creating dynamic frames. The goal is to ensure that characters maintain a recognizable appearance throughout, thereby reducing instances of characters phasing through objects or transforming in unexpected ways.

Moreover, the Gen-4 models are claimed to have an improved capacity for “understanding the environment” and simulating real-world physical interactions more effectively. The advantages of capturing real-life footage, such as shooting a bridge from multiple angles, contrast sharply with AI’s tendency to produce inconsistent renditions of the same structure—an issue that Runway is working to rectify.

For those interested, examining the demonstration videos from Runway reveals substantial improvements in consistency, though these selections are curated from a broader range of outputs. The characters featured in one particular clip exhibit a notable degree of uniformity across shots, despite minor differences in facial features, attire, and perceived age.