AI Video Generators Struggle with Simple Sequences
Veo and other AI video generators make it easy to bring your wildest visions to life, right?
While they can indeed be good for wild visions, they have a lot more trouble with mundane things you’d think would be easy. Sequences of specific actions tend to come out wrong. For instance, the AI will reverse the order, add unnecessary movements, or show results before causes.
If you’re making videos, this is useful to keep in mind, to keep you from putting too much time and too many credits down the drain, trying to make something work that AI simply isn’t capable of creating yet.
It’s doubly good to keep in mind if you’re starting out, and not too familiar with AI video behavior. In that stage it can be harder to distinguish whether the problem is lack of experience, or the AI itself.
I have a lot of examples in my AI outtake vault of video generations that screw up mundane subjects and get cause and effect backward.
To keep this blog from being 50 pages long I’ll focus here on three examples.
Stepping in a Puddle
To advertise residential walkway installation, I wanted to show someone with nice shoes stepping into a mud puddle, as if he didn’t see it until too late.
Here’s the prompt I used: “Close up of a man wearing dress shoes, socks, and pants walking along a worn grass trail in a yard. Without hesitation he steps into a muddy puddle, as if he didn’t see the puddle before he stepped into it, and continues walking.”
No matter how specific I got or how many variations I generated, it always put something extra into the act of stepping in a puddle that made it look strange.
Muddy Footprints
For a later scene I wanted to show muddy shoe prints that somebody had tracked into a nice house.
Here’s the prompt I used: “Close-up overhead shot of clean carpeted floor with fresh brown muddy shoe prints tracked across, dirt and water marks from shoes, indoor lighting.”
Instead of showing footprints being tracked in by someone walking, the AI just made them appear on the carpet out of nowhere. Result before cause.
You can see the finished walkway ad here.
Putting on a Backpack
For a different project, about staircases for waterfront properties, I wanted a simple shot of someone putting on a backpack.
Here’s the prompt I used: “Close-up shot of hands with short fingernails putting on a green canvas hiking backpack. Man wearing dark jeans and red flannel shirt. Professional lighting, indoor setting.”
I tried a bunch of versions and the AI kept having him fidget with the pack instead of putting it on, or put it on his stomach, or pick it up in some other unnatural way. Eventually I got a clip that was passable, but the backpack straps still phase through his body in an impossible manner.
Most people won’t notice so I used the clip, but it goes to show how a very simple action can be a tough nut to crack.
You can see the finished staircases ad here.
Takeaway
If you’re trying to generate something that involves a sequence of actions, and your AI is struggling, my advice is to set a cutoff for yourself and just pivot to something else if you don’t get what you want by your cutoff.
It’s worth exploring your ideas and not giving up too early, but at the same time, sometimes it really is the AI’s limits that are the issue.
Being able to recognize that will save you time and credits.
Struggling With an AI Video Project?
If you’re burning through credits and not getting what you want, contact me for a second opinion, or I may be able to take the whole thing off your hands. Click here to learn more about my AI work.