Expressive Speech-to-Face

This project aims to enable controllable generation of expressive, speech-driven facial animation in video games and multimedia. Current methods use procedural tools to animate mouth movements given speech clips, but they lack realism. This research proposes to develop a new approach for controllable speech-driven facial animations, combining the realism of recent mesh deformation methods with the ability to control the animation process. Using machine learning and a dataset of expressive facial performances, the research will focus on learning speaking styles, and the artistic manipulation of these styles, to achieve more nuanced and convincing animations. The anticipated outcome is improved facial animation quality in video games, benefiting the partner organization by providing a tool that enhances character expressiveness and overall gaming experience.

Faculty Supervisor:

Karan Singh

Student:

Partner:

Ubisoft Toronto

Discipline:

Computer science

Sector:

Information and cultural industries

University:

University of Toronto

Program: