Automatic analysis of sport videos is an attractive research area in computer vision that is driving the sport analytics towards a more technological edge. By automatically analyzing sport videos, lots of information could be drawn that benefits the teams, coaches, referees, players and even the fans, such as: extracting strategy of the game, technique and performance of each individual player, performance of the referee in a competition, and etc. This area of research, although attracted many researchers in computer vision community, is still in its infancy.