How The Irishman’s De-Aging Shows the VFX Pipeline Behind Digital Doubles

Film studios use AI-powered de-aging and digital face-replacement technology to make actors appear younger or place their likenesses on stunt doubles and body doubles. The 2019 film ‘The Irishman’ showed the scale: Netflix and director Martin Scorsese had Industrial Light & Magic digitally de-age Robert De Niro, Al Pacino, and Joe Pesci across multiple life stages—De Niro alone appearing as Frank Sheeran at ages 24, 36, 41, 42, 47, and 55, before his actual age of 76.

Unlike deepfakes made from uncontrolled internet images, production de-aging relies on high-resolution photogrammetry, performance capture under guild agreements, and supervised visual-effects work conducted frame by frame. The pipeline involves photogrammetry booths, 3D model construction, proprietary software that extracts performance data from lighting and texture, and visual-effects artist review—processes that create a clear chain of consent and accountability distinct from the unauthorized synthetic media that raised industry alarms.

The technology sits at an intersection of rising capability and rising stakes. Deepfake numbers increased tenfold from 2022 to 2023, and research has found that nearly 80 percent of viewers could not identify a deepfake even with content warnings. Meanwhile, studios continue to invest in de-aging as AI advances make the technology easier to produce, drawing attention from lawmakers and guilds seeking to protect performers’ likenesses.

Capture and 3D model construction

The process begins with photogrammetry: actors enter booths surrounded by hundreds of cameras, sometimes arranged in an orb shape, that capture thousands of overlapping high-resolution 2D images from multiple angles. For roles involving dialogue or varied expressions, performers require extensive facial scans showing different emotions and mouth positions. Simultaneously, separate motion-capture sessions record body movements as actors perform scenes in a studio.

Visual-effects artists convert the 2D photograph data into 3D digital doubles using techniques described as “akin to digital origami.” Overlapping images are mapped and folded in relation to one another based on camera coordinates. The resulting 3D model is rigged to a virtual “skeleton” for animation—a technical step that distinguishes this workflow from generative-AI-only approaches that lack the captured geometry and performance data.

A single digital head model costs approximately one million dollars to create. For ‘The Irishman,’ ILM developed an innovation that bypassed traditional photogrammetry for on-set work: a three-camera rig consisting of a primary camera flanked by two infrared cameras with light rings. The infrared cameras captured lighting data without being visible to the production camera, allowing the director’s normal lighting to remain undisturbed. This approach meant actors performed without facial markers, helmets, or tracking devices—preserving natural performance while generating the data needed for digital reconstruction.

De-aging by the numbers
A single digital head model costs approximately $1 million. Individual animated frames range from $30,000 to $100,000 depending on complexity.

De-aging and face transformation via machine learning

For de-aging, machine-learning models trained on high-resolution facial imagery transform an actor’s apparent age frame by frame. This training process relies on curated, permission-based imagery captured specifically for the project—fundamentally different from deepfakes scraped from uncontrolled internet sources with no authorized performance data.

On ‘The Irishman,’ ILM developed proprietary software called Flux (Face Lux) that operated without manual keyframe animation. Flux combined HDRI (high dynamic range imaging) and LIDAR data captured on film sets, solving for camera location, lighting intensity, color temperature, and infrared illumination data. The software then deformed digital doubles frame by frame to match the captured performances. Critically, as ILM explained the process: there are “not one facial animator listed in the credits—since Marty did not want to change the actors’ performances.” The algorithm handled the facial deformation; artists supervised the result.

ILM also built an AI Face Finder system using an extensive library of the three actors at targeted ages. Rather than generating faces directly, the system searched for matching eyes, angles, and lighting conditions in the reference library, providing a “sanity check” for rendered shots. The team supplemented this with 2D compositing solutions for body modifications, particularly in flashback sequences where the actors’ original performances were preserved while their apparent age shifted.

Individual animated frames range from $30,000 to $100,000 depending on complexity.

VFX supervision and how production differs from deepfakes

Production de-aging is not a fully automated process. Visual-effects supervisors review every frame, evaluating whether the machine-learning transformation looks photorealistic in context, whether the lighting sits correctly on the digitally altered face, and whether expressions match the actor’s performance.

The distinction between authorized production de-aging and unauthorized deepfakes rests on consent, disclosure, and technical rigor. A deepfake creates a convincing depiction of someone “saying or doing things they never did,” typically without permission or transparent acknowledgment of synthesis. Production de-aging operates under performer consent, published SAG-AFTRA rules, and frame-by-frame VFX oversight. Studios maintain exclusive use of curated training data, proprietary models, and supervised intervention—conditions that do not exist in deepfake pipelines, which operate on scraped images with no authorized performance capture and no artist oversight.

There is not one facial animator listed in the credits—since Marty did not want to change the actors’ performances.

SAG-AFTRA contracts and the regulatory landscape

A 2023 SAG-AFTRA agreement with the Alliance of Motion Picture and Television Producers (AMPTP) addresses AI for theatrical releases and streaming projects. It requires that studios obtain a performer’s consent before creating a digital replica of them, and that the performer is paid if the replica is used in their stead.

A separate SAG-AFTRA contract, the Interactive Media Agreement covering video-game performers, lets them “suspend consent for Digital Replicas during a strike,” effectively enabling their digital counterparts to join labor actions. The tentative three-year deal, approved by the SAG-AFTRA National Board in 2025 and submitted to members for ratification, would include compounded compensation increases of 15.17 percent, plus additional 3 percent increases in November 2025, November 2026, and November 2027.

Beyond guild contracts, the proposed NO FAKES Act (Nurture Originals, Foster Art, and Keep Entertainment Safe), introduced in 2024, would create a federal intellectual property right over voice and visual likeness. The legislation defines a digital replica as “a computer-generated, highly realistic electronic representation that is readily identifiable as the voice or visual likeness of an individual” and would grant post-mortem rights for an initial 10 years, renewable in five-year increments if the replica was used in the preceding two years, for up to 70 years after the individual’s death. The act carves out exceptions for documentaries and biographical content, satire, and parody, provided these uses are not “presented as though the fake is actually the real person.” Enforcement includes litigation penalties.

The regulatory patchwork reflects growing concern. China has required disclosure of AI-generated content since 2019; its 2023 “Deep Synthesis Provisions” update added a requirement for explicit consent from depicted individuals. The European Union’s AI Act, expected to enter into force in 2025, mandates disclosure of artificially generated or manipulated content. The United Kingdom criminalizes sharing deepfake adult content; South Korea has banned deepfake distribution outright. Most countries lack specific deepfake legislation, but the trend is clear: consent and transparency are becoming legal requirements, not industry norms.

Photo: Saiphani02 · CC BY-SA 4.0 · via Wikimedia Commons