Nov 15 / Christian Bull

Unreal's Metahuman and the Art of 3D Face replacement

The Best of Both Worlds: How Mastery of Fundamentals Makes Using Software Easier

How to Replace an Actor’s Face in VFX

There are many reasons why you might want to replace an actor’s face with something else in visual effects. Maybe you want to turn them into a creature. Maybe you want to add digital makeup, doing significant amounts of aging or de-aging. Quite often, you’ll have a stunt performer who needs to look like the actor. And then, of course, there is the Hollywood trend of digital necromancy; putting dead actors’ faces on stand-in bodies.

Whatever it might be, your options are:
  • 2D-based solutions (e.g., deepfakes) for impressive but limited results.
  • AI-based solutions with restricted control.
  • 3D solutions for full control.

I’ll go into Deepfakes and AI face replacement another time. Today, I want to go into 3D face replacement with Unreal’s MetaHuman, for reasons that will become clear.

The Hard Way vs. the Easy Way

These days, there’s a hard way and an easy way to achieve this. Let’s start with the hard way, and I’ll explain why it’s important.

The Hard Way: Step-by-Step Process

1. Scan your actor’s neutral face using photogrammetry. This can be taking photos on your phone, or using a professional photogrammetry service.

2.Scan them with various facial expressions. This follows a system called FACS, giving you a 3D representation of every muscle movement.

FACS gets it’s name from the fact that after an actor has been scanned in all 52 facial positions they always sigh “Oh for facs sake!” (I’ll get my coat…)



3. Create a rig to animate those shapes. When you lift the brows, the rig moves just like the actor, complete with forehead wrinkles.

4. Film your actor delivering their performance. This requires a head-mounted camera, which may need painting out post-take or can be done later without head movement.

5. Use facial motion capture software to connect your actor’s recorded performance to your rig.

6. Track the head of the actor on your filmed plate using your chosen take.

7. Have your motion-captured head follow that track.

8. Create 3D hair, if necessary; if you need hair to fall over your digital face, that’s going to have to be 3D.

9. Light your scene and render the face and any digital hair, then composite everything together.

The Reality Check

3D face replacement is a demanding process that can take months if you’re working alone. My visual effects company has invested over £10,000 in developing this pipeline, and larger companies may spend significantly more. Scorcese had to wait years to get enough budget to do it for the Irishman.

The Game Changer: MetaHuman

Here’s where Unreal’s MetaHuman comes in. This tool does almost everything above, for free, with much less effort—perfect if you have at least an intermediate understanding of VFX.

So…Why Did I Just Waste Your Time Explaining the Hard Way?

  • I said the MetaHuman does almost everything. When you’re in the trenches, you’ll hit a thousand problems that you’ll need to figure out for yourself. For example, what if you’re replacing a stunt double with an actor’s face, but the stunt double has a huge head, or a tiny neck, or something that just means that the digital face can’t fit onto the stunt actor’s body?
  • The end result is that there’s a MASSIVE gulf between knowing how to use the software and actually using it for shots, in the same way that knowing how to put your foot on the gas is different to knowing how to drive, which is different to understanding how a car works. When your car breaks down in sub-zero temperatures, knowing how to put your foot on the gas isn’t going to help much.
  • Filmmaking is problem solving from day 1. You normally break down in sub-zero temperatures on day 2. If you can solve creative AND technical problems, you’re set. The more that technology does for you, the more it robs you of the opportunity to improve this core skill.

Key Takeaways
  1. Technology moves quickly, but that’s great - If it didn’t VFX and digital film making wouldn’t be accessible to everyone, and they are!
  2. Superficial skills evolve, but understanding is timeless - While technologies change, the core knowledge behind them remains valuable. Visual storytelling is done differently these days, but the core concepts can be learned with a pen and pencil - that’s why so many directors of animated feature films start in the storyboard department! Learning the basics gives you a solid foundation, ensuring you’ll always have the essential skills to adapt. You’ll never be left behind if you understand how and why things work-you’ll always be ready to learn and grow.
  3. True creative freedom comes from understanding - Technology can assist, but it can’t replace your judgement. And your judgement comes from hard won experience. We can all create beautiful images and shots with AI. A 5 year old can create something that just a few years ago would take a team of people. But how, where, and why to use that something? That’s where your judgement comes in. And you earn your ability to judge through hard work.

So get working, and kick it over to us so that we can help you out by providing feedback!


Want a video tutorial?

Thanks for reading, I hope you’re as excited as we are to see our hard work come to life in The Burning Girls. If you’d like a video lesson going through the process in Blender, drop me a line, and if you haven’t checked out our recent “Bagman” demonstration, it’s a great companion to the techniques discussed here and can give you even more insight into our process.

Stay tuned for upcoming demos, tips, and behind-the-scenes looks at our work. Until then, happy streaming and creating!