Controllable and editable character performance using Implicit Neural Representation approaches

Nowadays, many of the movie characters whose performances move us on screen are at least in part digital. From superhero stunts to de-aged beloved actors and actresses, visual effects artists have to create digital characters and painstakingly reproduce performances to convince audiences. New Deep Learning (DL) technologies are emerging to help alleviate the processes. For instance, Deep Fakes have been quite successful at swapping facial performances. Other promising approaches are emerging under the large umbrella of Implicit Neural Representation (INRs) such as Neural Radiance Fields (NeRFs). We wish to explore novel ways to automate parts of the workflows involved in creating so-called Digital Doubles using NeRF-like algorithms.

Faculty Supervisor:

David Lindell

Student:

Partner:

DNEG

Discipline:

Computer science

Sector:

Information and cultural industries

University:

University of Toronto

Program: