Facial expression identification over a time series of images

Facial expression is a universal language to convey emotions and significantly affects social interactions. While psychologists have investigated facial expressions for decades, they have recently found their way into human-computer interactions and the gaming industry. A lot of research has been published on automatic detection of human emotions given either a single image or a series of images. In this project, we propose a new method for facial expression interpretation over a time series of images. We will first identify key facial expressions utilizing convolutional neural networks (CNN) and long short-term memory (LSTM) methods. Then, we will interpolate between these key facial expressions utilizing artificial intelligence while tracking head, eye, mouth, and eyelid. Finally, we will classify the emotions conveyed by the facial expressions over the entire time series of images. We expect to improve the state-of-the-art performance with this approach. Furthermore, we think that this approach would be more easily conveyed to the animation and gaming industries.

Faculty Supervisor:

W. Robert J. Funnell

Student:

Majid Soleimani

Partner:

SeekShift

Discipline:

Engineering - biomedical

Sector:

Information and cultural industries

University:

Program: