Debugging ML via Feature-guided Analysis: Analyzing Neural Network Robustness

Neural Networks (NN) use a set of individual units (neurons) connected together to learn a specific behavior from a dataset. For example, NN excel in classification tasks where given a dataset labelled with presence or absence of a feature in each entry, they are able to detect the feature presence on new inputs. This technology has been applied in many fields, including automotive, aerospace, medical and others.
A significant drawback of NNs is that their behavior is unknown, since it only depends on the training data and it is not understandable to humans. This significantly limits their applicability, especially in safety-critical fields. For this reason, NN interpretability is a growing field of research. Feature-Guided Analysis (FGA) is a technique that extracts rules from NNs and can help explain how the values assumed by individual neurons affect the outcome of the network.
This project aims at replicating the results of this technique on a new image-based dataset and improve upon the limitations of the existing technique. We further aim at employing this technique to analyze the robustness of the network, providing engineers with a description of which portion of the NN are more susceptible to perturbations on the inputs.

Faculty Supervisor:

Mark Lawford

Student:

Partner:

University of Bergamo

Discipline:

Computer science

Sector:

Education

University:

McMaster University

Program: