Ecological Interaction Perception in Multi-Talker Environments

The process of identifying someone is talking to you and listening to that talker is one of the most complicated tasks people perform every day. This task is still not well-solved for robotics and it represents one of the fundamental barriers to effective human-robot interaction. For both humans and robots, several social cues must be observed and processed, and then natural language understanding performed to first determine that a speaker is talking to an observer. Next, the observer must attend to and recognize what a speaker is saying. The proposed project will break this process down into individual components and attempt to solve it using a combination of pre-existing and novel techniques to 1) identify talkers in an environment, 2) determine which if any talks are addressing the observer, and 3) perceive and recognize the speech of talkers addressing the observer. With the proposed project, we aim to produce a robotics system capable of identifying when speech is directed at the system, and then perceiving that speech in a natural environment.

Faculty Supervisor:

Matthew Tata

Student:

Partner:

Istituto Italiano di Tecnologia

Discipline:

Computer science

Sector:

Education

University:

University of Lethbridge

Program: