Agent ClickBait Attacks

ServiceNow develops a platform for client organizations to manage and automate large-scale processes across various industries. In 2020, ServiceNow acquired Element AI to strengthen its presence in the Artificial Intelligence (AI) research landscape and the Canadian AI ecosystem. This acquisition enabled the development of AI-driven products that improve the platform’s capabilities. ServiceNow Research has made significant contributions to the field of foundation models, notably in Natural Language Processing (NLP), and has a strong presence in developing generative models for different data domains.
This project addresses the risks posed by visually deceptive or manipulative stimuli to autonomous vision-language agents, which can lead to subtle, context-dependent errors in perception and reasoning. Unlike traditional pixel-level attacks, clickbait-style manipulations exploit cognitive biases, threatening the reliability and security of agents in sensitive domains. By analyzing these vulnerabilities and developing defences, the project aims to enhance agent robustness, ensuring safe deployment of vision-language agents.
This project supports the deployment of vision-language agents designed for use in safety-sensitive areas. Currently, user-interface (UI) agents are mainly restricted to HTML; limiting the scope of automated task completion. The results of this research will be made open source to promote further studies in this field.

Faculty Supervisor:

Gauthier Gidel

Student:

Partner:

ServiceNow Canada

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

Université de Montréal

Program: