In 1907, Norman Dawn mounted a sheet of glass a few feet in front of the camera and painted the missing parts of the world onto it, in the process establishing one of the most powerful visual tools in film, the matte painting. As with everything in film, there was a creative limitation (how convincingly can you paint a photoreal background), and a technical one (if you move the camera at all, the illusion breaks).

Those creative and technical constraints still exist in some way today. Painting skill will massively help your decision making, and even though we can move the camera now, there is still a limit to how much you can move it, and to what moves in frame (matte painted trees work great until there is wind).

As film moved digital, so did matte painting, picking up the name DMP (Digital Matte Painting). This changed matte painting in a few huge ways. Firstly, photos became a key part of the painter's tool set. Secondly, camera movement became easier to fake, using a 2.5D approach. Instead of a single painting, a DMP could be several paintings at different depths to the camera. If you move the camera, a painting close to the camera moves more quickly than a painting further away, just as when you move your head, objects closer to you move across your field of vision more quickly than those in the distance.

Whilst this approach has its limits (you cannot move the camera too much, or the illusion breaks, and it breaks very quickly close up; also reflections and transparency change depending on the camera position, and they do not change in a painting), it is still widely used today, and has the huge advantage of not needing heavy renders.

Cards are great, but they have real limits, especially for objects close to camera, where the two dimensionality becomes apparent very quickly. 3D does not have that problem, but building a whole environment is massively more labour intensive than a painting, and comes with heavy render times.

Projecting onto 3D geometry gets you the best of both worlds. There is no magic to digital projection, it works the same as projection in real life: you can only project down from a single angle. So if the camera moves to a new position and reveals the side of an object that is not getting any projection information, the illusion breaks. Buuuut you can set up another projection, from a different angle, to cover that.

It is a very powerful technique, but of course it still has its limits (again, reflections, transparency, translucency and, of course, movement will all break the illusion quickly). That just means you are not going to use matte painting for every goddamn shot, just when it is an appropriate tool. If full 3D is the only way, then it is the only way. If render time is a concern, it is fairly common to do full 3D renders, but instead of rendering every frame, you just do the extremes of the camera movement and project them down.

I do not care what AI can do or where it goes, you are never going to convince me that learning painting skills is a waste of time in filmmaking. But AI definitely can do a great job of handling some of the hardest parts of matte painting, especially how it handles light and shadow. It is pretty unreliable when it comes to perspective, so 3D geometry can be useful as a guide.

As with everything in film, AI can give you access to better or faster results, but needs a solid understanding of the fundamentals.

You will need to know when, why and how to use these various matte painting techniques. When you understand that, integrating AI becomes much more simple.

Try it on the demo above

Hit AI Painting and scrub through the camera move. This shows you the pros and cons of projected matte painting at a glance. The massive advantage is that you can move the camera and the painting comes to life. The downside is that you cannot move it far before the illusion breaks.

That is where you would start adding more projections: one for the foreground buildings, one for the background buildings, one for the road, and so on. Hit AI Painting again to put the original back.

Then make your own

The download geometry still button gives you a clean render of the bare geometry, shot from the projector. Take it to an AI image editing model and prompt something like "Create a photoreal city with the exact composition and proportions of this rendered geometry", plus whatever else you want: bright daylight, cyberpunk feel, fill your boots.

Keep the framing and the horizon exactly where they are, or it will not line up. Then Load your image to project it onto the same geometry and run the same move.