Background
Graph Neural Networks (GNNs) are a class of machine learning algorithms designed to process and learn from graph-structured data. Their primary goal is to learn meaningful node, edge, or graph representations (embeddings) by aggregating information from neighboring nodes. GNNs can be viewed as a generalization of convolutional neural networks to non-Euclidean domains, where data is represented as graphs rather than regular grids, such as images. GNNs have achieved remarkable success in tasks such as node classification, link prediction, graph classification, and recommendation systems.
Residual connections, which allow nodes to retain information from their initial (or previous) embeddings throughout the learning process, have been shown to significantly improve the performance of many GNNs. However, the impact of residual strength on prediction error and generalization performance (i.e., the extent to which a trained model can generalize to unseen data) remains largely unexplored. This project is dedicated to understanding the relationship between residual strength in GNNs, prediction error, and generalization bounds. This leads to the following research questions:
- How should the residual strength be selected to control the classification error on graph data?
- How does the residual strength affect the generalization bound of GNNs with residual connections?
The methodology combines both empirical experiments and theoretical analysis. On the experimental side, we will train some GNNs with varying residual strength coefficients on standard graph datasets (such as Cora and Citeseer, etc) and measuring classification error and comparing performance against baselines. On the theoretical side, we will analyse how generalization bounds change when a residual strength parameter is introduced, with the goal of identifying conditions under which stronger residuals either tighten or loosen the bound.
Requirements
Solid foundation in linear algebra and calculus, experience with Python and PyTorch.
Related References
- Predict then propagate: Graph neural networks meet personalized PageRank, ICLR 2019
- Simple and deep graph convolutional networks, ICML 2020
- Adaptive Initial Residual Connections for GNNs with Theoretical Guarantees, AAAI 2026
Contact
Supervisor: Ahad N. Zehmakan
Email: ahadn.zehmakan@anu.edu.au.com
If you are interested, please write me an email, including (1) what aspects of this project interest you the most, (2) what type of research project you are looking for, 6-unit, 12-unit, or 24-unit, (3) a copy of your transcripts and/or CV, (4) any questions you may have.