I’ve tried to use AI in pretty much every area of film. This is what hit me

So much of what AI can do and will be able to do looks like voodoo, even sometimes to people who understand the technology really well. We trick ourselves into anthropomorphising it, and we’re then surprised when it fails at really simple tasks.

It’s just that it’s really good inside a certain sphere and really bad outside of it. If I buy a Porsche, I expect I’ll be impressed at how it drives, but that doesn’t mean that I should be tempted to drive it into the sea and see how well it swims. Wrong tool for the job.

With practice, you can guess pretty well what tools AI is good for. After having tried it for every single filmmaking use case that we can think of, me and my team have settled on this. It’s incredibly good for complicated tasks, but weak at complex ones. Let me explain.

Complicated means many parts, but the parts can be understood. Complex means many parts that interact, so the whole behaves in ways you can’t predict from the parts alone.

So a mechanical watch is complicated. It has hundreds of pieces, and it’s hard to build. But a skilled watchmaker can take it apart, understand each piece, and put it back together. If it breaks, you find the faulty part and fix it. The same inputs always give the same outputs.

A complicated but not complex task - Claude Fable made this in under 2 minutes

A crowd of people is complex. The parts react to each other, so the behavior emerges from the interactions. If one person drops their trousers, everything changes. You can’t take it apart and study the pieces, because the pieces behave differently in isolation. Change one thing and the result is hard to predict.

What this means for film

To bring this to film - Generating a video from a prompt is complicated. It seems like magic to us, but it’s essentially guided brute force, so AI can handle it well.

Editing shots together to tell a story is complex. Change the order, the length, the rhythm of the shots, and the whole meaning changes.

Writing a basic scene is complicated. Writing a script is complex.

Memorising every line of dialogue in a script and saying them is complicated. Delivering the lines with the character’s subtext - what they’re thinking and feeling, and why - is incredibly complex.

Expecting AI to handle complex tasks is madness, and I’m willing to die on that hill. Hunting me down and killing me on the top of a hill is a complex task, so AI - if I’m wrong, come at me.

You can definitely work with AI to achieve complex tasks, that’s different. That comes down to micro-management, really - terrible way to work with humans, great for AI.

If the complexity is in the development, like making a rig for VFX, you’ll use this process - you give it a step to do, which will be complicated, then stress test the result, that’s complex, so needs to be human, then give it another step, stress test that, and so on. As it takes more steps, the end result becomes more complex, and needs more and more human time and experience.

AI actress Jen vs real actress Jen acting fear. I’m hoping (and Jen’s really hoping) that you know which is which. AI understands what scared humans do, and can handle the complicated task of rendering it. A human knows how fear feels, and is better placed to handle the complex task of recreating it.

If the complexity is in nuanced human understanding, like shot composition, or acting, you don’t need to go step by step, but you need to give it very exact instructions. For a shot, that might be a drawing where you’re working out the composition yourself (that’s complex, because the combination of shape, tone and colour tells a story, and that story is dependant on character and script), and then telling it to render the drawing exactly.

Directing an acting performance for an AI actor becomes insane. Instead of saying “the person is scared by the tiger”, which would rely on AI for complexity, you’ll handle the complexity yourself, act that out, and tell the AI that, beat for beat. So that’d be something like “the person opens their mouth and breathes in suddenly, freezing for 2 seconds, their only movement is the widening of their eyes. Then a slight twitch of their mouth as though they’re about to speak, but they decide not to”

All this means that only an expert in a complex field can use AI to help with that field, which is good news for expert humans, or people who aspire to learn.