Investigating the effect of a novel regularization technique for neural networks

Despite the fact that neural networks have been widely applied in practice, training such networks can suffer from slow convergence, poor local minima and some other difficulties such as catastrophic forgetting. Such shortcomings severely undermine the applicability and usefulness of neural networks. The objective of this project is to identify the reasons behind such difficulties in training and to investigate the effect of novel regularizations. We will analyze the strengths and shortcomings of existing algorithms and benchmark recently proposed regularizers for neural network training. The performance of these techniques will be evaluated for supervised tasks.

Faculty Supervisor:

Dale Schuurmans

Student:

Partner:

Royal Bank of Canada (Borealis)

Discipline:

Computer science

Sector:

Technology; Information and Communications Technology

University:

University of Alberta

Program: