Interpretation and Characterization of Recurrent Neural Networks through Lyapunov Exponent Methodology

Neuroscience-inspired AI has emerged as state-of-the-art in many machine learning applications. Recurrent Neural Networks (RNNs) are a machine learning tool used to learn patterns in sequential (time-dependent) data which have also been used to model neural dynamics in the brain. Various frameworks have been developed to create RNNs capable of learning from data which have long-term dependencies. Architectures such as the LSTM and GRU have been shown to successfully learn long-term dependencies, but the underlying mechanisms which lead to their success and failure are not well understood. By incorporating tools from dynamical systems, we will examine the learning trajectories of different networks as they learn different tasks. This will create a general and formal structure in which the dynamics of different networks and models can be compared. Understanding the dynamic properties of these systems will support and broaden the performance and range of RNN-based applications of machine learning to complex, dynamic processes.

Faculty Supervisor:

Guillaume Lajoie

Student:

Partner:

University of Washington

Discipline:

Computer science

Sector:

Education

University:

Université de Montréal

Program: