Exploring Kolmogorov-Arnold Networks (KANs) for Materials Informatics Applications

Are KANs better than MLPs for real-world applications?

Picture of amanda-barnard.md Amanda Barnard AM

9 May 2024

Based on Kolmogorov-Arnold representation theorem, Kolmogorov-Arnold Networks (KANs) have been proposed as superior alternatives to Multi-Layer Perceptrons (MLPs). Unlike MLPs KANs have learnable activation functions on the weights (not neurons), and every weight is replaced by a univariate spline. For mathematical physics applications it has been shown that shallow KANs can achieve comparable or better accuracy than deeper MLPs, have faster neural scaling laws, and greater interpretability. At this stage little is known about how KANs perform on more complex tasks, and although they are proposed to be suitable for scientific discovery, they have not been rigorously tested on scientific data sets suffering from common problems such as data scarcity, sparsity, imbalance, high-dimensionality and non-linearities such as bimodal or trimodal distributions.

Research Questions and Tasks

This project aims to directly compare KANs and MLPs for a series of scientific machine leaning tasks, testing the claims of superior performance. The computational requirements and scaling will also be considered, resulting in a definitive decision framework for choosing when to use KANs, based on resources, time, task, need for interpretability and properties of a data sets. Applications in materials informatics are used as an exemplar, representative of challenges experienced in most areas of computational science.

References

  • https://arxiv.org/abs/2404.19756
  • https://arxiv.org/abs/2007.15884

Requirements

Background and experience in Machine Learning (i.e. COMP3670/4670/4660/4650, STAT3040/4040). Experience with Python.

You are on Aboriginal land.

The Australian National University acknowledges, celebrates and pays our respects to the Ngunnawal and Ngambri people of the Canberra region and to all First Nations Australians on whose traditional lands we meet and work, and whose cultures are among the oldest continuing cultures in human history.

arrow-left bars search times arrow-up