Exploring AI Explainability Opportunities for Developing and Deploying Trustworthy ML Models

Protecting artificially Intelligent systems (AIS) is a demanding task Until recently, ML systems were not able to take an active role in general purpose or commercial applications, due to the lack of applicable algorithms, tools, hardware resources and most importantly training data. With the advancement of the internet, an abundance of training data is now available and has accelerated the rapid development of AIS, e.g., Chatbots using Large Language Models, image classifications, autonomous driving, etc. However, these technologies have their own limitations and risks. There is a growing concern about adversarial resistance of the AI technologies, unbiasedness, and inexplicability of the decisions AISs are making. These concerns largely fall under the AI Governance (AIG) umbrella. The sole purpose of AIG is to make AI technology more trusted, explainable, robust, and protect it from unwanted changes. TrojAI is working actively to address these issues and developing a solution to achieve better governance. This research will focus on how AI models can be better explained and prevent adversarial attacks. It will investigate, implement available model explainability, visualization techniques, and propose, develop, or modify new ones if required. It will also investigate and develop new AIG tools for feature engineering and performance monitoring.

Faculty Supervisor:

Kenneth Kent

Student:

Partner:

TrojAI

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

University of New Brunswick

Program: