Metamorphic Testing of Deep Neural Networks

Nowadays, we are witnessing a wide adoption of Machine learning (ML) models in many software systems. They are even being tested in safety-critical systems, thanks to recent breakthroughs in deep learning and reinforcement learning. Bombardier, a world leader company in the domain of aerospace have started integrating deep learning models in its products, systems and processes design used for certification and homologation. Although these ML models can reach high accuracy within their training data distributions, recent studies have shown that they can be very brittle. Hence, they should be tested adequately. However, because of the absence of a reference oracle, it is very challenging to reason about the behavior of a Deep Neural Networks (DNNs) and hence to test it. Novel testing techniques are needed both during model engineering and deployment phases, to guarantee the reliability and robustness of in-production DNN-based software. During the model engineering phase, developers need to assess the impact of their configuration choices carefully. The effectiveness of this assessment depends on the capability of testing data to trigger both the major functionalities of the model (regular cases) and the minor functionalities (corner cases).

Faculty Supervisor:

Foutse Khomh;Giuliano Antoniol

Student:

Houssem Ben Braiek

Partner:

Bombardier Aerospace

Discipline:

Engineering - computer / electrical

Sector:

Aerospace and defense

University:

École Polytechnique de Montréal