Based on Kolmogorov-Arnold representation theorem, Kolmogorov-Arnold Networks (KANs) have been proposed as superior alternatives to Multi-Layer Perceptrons (MLPs). Unlike MLPs KANs have learnable activation functions on the weights (not neurons), and every weight is replaced by a univariate spline. For mathematical physics applications it has been shown that shallow KANs can achieve comparable or better accuracy than deeper MLPs, have faster neural scaling laws, and greater interpretability. At this stage little is known about how KANs perform on more complex tasks, and although they are proposed to be suitable for scientific discovery, they have not been rigorously tested on scientific data sets suffering from common problems such as data scarcity, sparsity, imbalance, high-dimensionality and non-linearities such as bimodal or trimodal distributions.
Research Questions and Tasks
This project aims to directly compare KANs and MLPs for a series of scientific machine leaning tasks, testing the claims of superior performance. The computational requirements and scaling will also be considered, resulting in a definitive decision framework for choosing when to use KANs, based on resources, time, task, need for interpretability and properties of a data sets. Applications in materials informatics are used as an exemplar, representative of challenges experienced in most areas of computational science.
References
- https://arxiv.org/abs/2404.19756
- https://arxiv.org/abs/2007.15884
Requirements
Background and experience in Machine Learning (i.e. COMP3670/4670/4660/4650, STAT3040/4040). Experience with Python.