From Sports to Physics: Deep Representation Learning in Real World Problems

Abstract

Machine learning has recently made significant progress due to modern neural network architectures and training procedures. When neural networks learn a task, they create internal representations of the input data. The specific neural network architecture, training process, and task being addressed will influence the way in which the neural network interprets and explains the patterns in the data. The goal of representation learning is to train the neural network to create representations that effectively capture the overall structure of the data. However, the process by which these representations are generated is not fully understood because of the complexity of neural network data manipulations. This makes it difficult to choose the correct training procedure in real world applications. In this dissertation, we apply representation learning to improve the performance of neural networks in three different areas: NBA movement data, material property prediction, and generative protein modeling. First, we propose a novel deep learning approach for predicting human trajectories in sporting events using advanced object tracking data. Our method leverages recent advances in deep learning techniques, including the use of recurrent neural networks and long short-term memory cells, to accurately predict the future movements of players and the ball in a basketball game. We evaluate our approach using data from the NBA's advanced object tracking system and demonstrate improved performance compared to existing methods. Our results have the potential to inform real-time decision making in sports analytics and improve the understanding of player behavior and strategy. Next, we focused on group activity recognition (GAR) in basketball. In basketball, players engage in various activities, both collaborative and adversarial, in order to win the game. Identifying and analyzing these activities is important for sports analytics as it can inform better strategies and decisions by players and coaches. We introduce a novel deep learning approach for GAR in team sports called NETS. NETS utilizes a Transformer-based architecture combined with LSTM embedding and a team-wise pooling layer to recognize group activity. We test NETS using tracking data from 632 NBA games and found that it was able to learn group activities with high accuracy. Additionally, self- and weak-supervised training in NETS improved the accuracy of GAR. Then, study an application of neural networks on protein modeling. Recent work on autoregressive direct coupling analysis (arDCA) has shown promising potential to efficiently train a generative protein sequence model (GPSM) to adequately model protein sequence data. We propose an extension to this work by adding a higher order coupling estimator to build a model called autoregressive higher order coupling analysis (arHCA). We show that our model can correctly identify higher order couplings in a synthetic dataset and that our model improves the performance of arDCA when trained on real-world sequence data. Finally, we study material property prediction. Incorporation of physical principles in a machine learning (ML) architecture is a fundamental step toward the continued development of AI for inorganic materials. As inspired by the Pauling’s rule, we propose that structure motifs in inorganic crystals can serve as a central input to a machine learning framework. To demonstrate the use of structure motif information, a motif-centric learning framework is created by combining motif information with the atom-based graph neural networks to form an atom-motif dual graph network (AMDNet), which is more accurate in predicting the electronic structures of metal oxides such as bandgaps. The work illustrates the route toward fundamental design of graph neural network learning architecture for complex materials by incorporating beyond-atom physical principles.