Learning from relational data is crucial for modeling the processes found in many application domains ranging from computational biology to social networks. In this project, we propose to work on developing new modeling techniques that combine the advantages of the approaches found in two fields of study: Machine Learning (through graph neural networks and transformer networks) and Statistical Learning (through statistical relational learning methods). By combining the advantages of both approaches, we aim to obtain better prediction results for an array of problems such as classification and link prediction in relational data.