Real-Time Facial Animation Playback with Shader-Embedded Neural Networks

Creating lifelike digital characters is essential for multimedia and video games, with realistic facial animations playing a crucial role. This requires playing high-quality facial animations in real-time. Many existing methods either compromise on quality, require too much computing power, or use excessive memory. Our research aims to develop new techniques that ensure high-quality facial animations can be played back efficiently in real-time without losing detail. This project will research machine learning models that operate on each vertex of facial meshes independently, allowing these animations to be processed quickly using the existing pipeline of graphics cards. This approach leverages the parallel processing capabilities of graphics cards to achieve real-time playback. These methods will be tested on a large set of facial animations, measuring success through technical metrics and feedback from professional animators.

Faculty Supervisor:

Noam Aigerman

Student:

Partner:

Ubisoft Divertissement

Discipline:

Computer science

Sector:

Information and cultural industries; Manufacturing

University:

Université de Montréal

Program: