Robustness of Reinforcement Learning to Attacks and Adversarial environments

Reinforcement learning algorithms are successfully employed in diverse industries, for instance in autonomous driving, trading or gaming. However, their generalization especially in critical-decision making systems has raised concerns about their robustness to attacks. The 22nd recommendation issued by the White House (2016) to prepare for the future of AI states: Agencies (…) should ensure that AI systems and ecosystems are secure and resilient to intelligent opponents. Existing works on the robustness of RL to attacks assume that the attacks originate from an outsider (akin to a hacker) capable of manipulating the environment, which destabilizes the agent’s and degrades their performance. In this project, we adopt a setting where the attacker is naturally present in the environment. Their goal is to observe and model a targeted agent’s actions in order to destabilize and degrade their performance. This setting unveils the need for methods enabling humans to accurately interpret the intentions underlying the interactions between RL agents in order to identify potential threats and the need to develop new defense mechanisms to decrease the likelihood and the impact of such attacks.

Faculty Supervisor:

Audrey Durand

Student:

Partner:

Thales Canada Inc (Montreal, QC)

Discipline:

Computer science

Sector:

Management of companies and enterprises; Manufacturing; Professional, scientific and technical services

University:

Université Laval

Program: