Learning Discussion Thread Representations to Empower Conten-based Recommendation
VerticalScope is a company that owns online forums in many domains, such as automotive, health, technology, and powersports. VerticalScope uses a content based recommender system to mitigate the cold start problem, where a large portion of traffic on the forums are made by unregistered users. The goal of this project is to learn representations of discussion threads. Thread representations that capture semantic and contextual information can improve the recommender system to suggest more relevant threads to users, and boosts search engine optimization and user retention rate. Understanding user sentiments also allows for the discovery of trending topics, and personalized homepage and advertisements.
Learning thread representations has various challenges. Within the automotive forums for example, there may be multiple threads talking about buying and selling cars. However, though these threads may have similar context, the object of discussion (e.g. the specific car model) can be different, and the learned representations should capture these differences. Another related problem is that there are many out of vocabulary words that may be very important to the relevancy between two threads (e.g. the name of a specific product).