How AI text-to-video could expand marketers' digital toolkit

With AI tools such as DALL-E and Midjourney, users can generate a highly descriptive image with nothing more than a line of text. But what if you could take the same simple input and generate a dynamic video? This is the goal at the heart of a race between Meta, Google and tech startups looking to build the next must-have AI tool. And marketers are already dreaming up innovations that text-to-video AI will bring to their work. Check out the latest Ad Age events and award programs The space heated up this week with the announcement of a new video AI platform from startup Runway Research, which will roll out to limited audiences in the coming weeks. The model, dubbed “Gen-2,” will enable users to produce a three-second clip of original footage from a line of text, such as “late afternoon sun peeking through a New York City apartment window.” Gen-1 launched earlier this year and is purely video-to-video, meaning pre-existing video can only be edited, not created from scratch. Trailer for Runway's “Gen-2” model The quality of text-to-video content is currently nowhere near production-ready. But in time, the method may advance the digitization that marketers are already applying to their creative, said Luke Hurd, director of experience design at VMLY&R. Cars featured in TV commercials, for example, are often the product of computer-generated imagery (CGI), because using a real model presents all kinds of difficulties, from transportation to video editing. AI could similarly digitize other burdensome details in video, such as the background landscape and potentially the entire spot itself. Related: A generative AI guide for brands Text-to-video AI can expand marketers' generative AI toolkit should platforms be easy to use and integrate, as with the popular text-to-text model ChatGPT. There are potential downsides, of course, such as AI biases that could infiltrate video content and computing inefficiencies. The next six months to a year will be a major test for these models to work out their kinks. Tim Hawkey, chief creative officer at agency Area 23, has experimented with Runway’s Gen-1 and is eager to get his hands on Gen-2. He views text-to-video models as relieving certain quirks of text-to-image models. The “flipbook effect,” for example, refers to how anyone wishing to simulate a video with the help of an image generator will need to create hundreds of different images and then splice them together. The final product functions similarly to a flipbook. "Flipbook effect," as shown in a video created by Tim Hawkey for his YouTube channel Since text-to-video generators are trained on actual video content, however, creations have frame-to-frame consistency. The result is higher-quality, more realistic production, which many marketers will likely favor as a means of producing creative, rather than what is effectively stop-motion animation. It is unsurprising that tech giants have already dove head-first into the AI video niche. Meta last September introduced “Make-A-Video, a text-to-video generator that produces clips no longer than five seconds. Example prompts on the platform’s landing page include “A golden retriever eating ice cream on a beautiful tropical beach at sunset, high resolution” and “Humans building a highway on mars, highly detailed.” Google has unveiled two text-to-video models: “Imagen Video,” which looks very similar to Runway’s Gen-2 and Meta’s “Make-A-Video,” and “Phenaki,” a model aimed at creating longer videos based on more descriptive prompts. This latter tool is exploring what will likely be the future of text-to-video AI generation, since short videos will ultimately have to be spliced together, like images, in order to form longer content. Example prompt and video from Meta's “Make-A-Video” tool Both Meta and Google’s models are mainly research projects, and neither company has released details about when they might be released to audiences. Moreover, marketers hoping to soon integrate any kind of text-to-video content will need to curb their enthusiasm, said Paul Roetzer, founder and chief executive of the Marketing AI Institute. It is clear just from looking at the outputs produced by Runway, Meta and Google’s models, that AI-generated videos are of poor quality. They seem to have an especially hard time simulating motion, such as dancing and walking. AI marketing blog How brands are using ChatGPT, DALL-E 2 and other tools Read more here And while image generators have advanced leaps and bounds compared to their rollouts a year ago, Roetzer believes the learning curve is steeper for video AI. Image and text generators produce a single, static output, whereas video generators produce something much more dynamic. Dealing with issues of continuity and consistency require quite a bit of work, said Roetzer. Still, if the explosiveness of generative AI has taught us anything, it is that one should not bet against the abilities of these models. As Hawkey said: “Gen 2 is a bit of a novelty, but the technology is a breakthrough.”

How AI text-to-video could expand marketers' digital toolkit

Featured Stories