Towards understanding people’s interactions with a video question-answering AI system
In recent years, there has been a rise in the development of interactive AI systems and applications where the user follows a procedure to complete a task, including (instructional) fitness and cooking apps to name a few examples. Most of these efforts focus around video question-answering (QA), where a system is built to answer questions a user asks when watching an instructional video. However, little is understood of the usability of these interactive AI technologies and what types of questions a user might ask in a realistic scenario, when they are following the procedure, in an instructional video.
In this project we aim to observe the user in real time, where they are actually asked to follow a procedure demonstrated in a video, and interact with it via asking questions. Our goal is to understand the nature of the questions that users would ask, and whether they would find it beneficial to be able to interact with such an AI system via natural language queries.
Khai N. Truong
Samsung Electronics Canada
Computer science
Manufacturing
University of Toronto
Accelerate
