Deep Sensor Fusion of Appearance, Depth and Thermal Camera Features for Autonomous Driving

The purpose of the proposed research is to explore a deep sensor fusion framework for autonomous driving using a thermal camera and stereo camera. This is a useful driver assistance application for challenging environments such as night-time and rainy or snowy weather. Single sensor, such as monocular, stereo and thermal camera-based autonomous driving applications are not robust enough for practical applications. The monocular camera which provides descriptive appearance information is affected by illumination variation. This can be addressed by utilizing stereo-based depth information. However, the stereo depth is less reliable at nighttime. This issue can be addressed by using the thermal camera. It can be seen that features obtained from the thermal camera and stereo camera are complementary, resulting in the increase of robustness of autonomous driving applications. In our previous research we effectively fused stereo-based depth information and monocular camera-based appearance information using the novel ChiNet. TO BE CONT’D

Faculty Supervisor:

Zheng Liu

Student:

Partner:

Toyota Technological Institute

Discipline:

Engineering

Sector:

Automotive; Other

University:

The University of British Columbia - Okanagan

Program: