Intelligent musical instrument projects with IMPSy

Picture of Charles Martin

6 May 2024

An embodied intelligent musical instrument

The idea of this project is to develop new kinds of intelligent musical instruments and tools for designing them, this is a multi-year research effort led by Charles Martin with many sub-projects and involving PhD, Master, Honours, and undergrad students. It would be great to have you involved in a subproject if you would like!

We define an intelligent musical instrument as:

An instrument where an AI system generates actions independently of a musician’s either reacting to the musician’s actions or not.

So far within this research project, we have created IMPSy, a system for creating neural network models for real-time musical interaction. We have embedded IMPSy in several musical instruments such as EMPI, with MIDI controllers and Pd, and hardware synthesisers.

At the moment we are focussed on:

  • deploying IMPSy reliably on Raspberry Pi computers
  • improving the experience for collecting and curating data when using IMPSy
  • making IMPSy more configurable and useful for regular musicians
  • developing new intelligent musical instrument prototypes with IMPSy (let’s make an orchestra of intelligent musical systems!)
  • creating workshop activities using IMPSy on embedded systems
  • improving the inference speed and runtime experience

Any of these topics could be the start of a short or long project.

As part of this project, you will conceptualise, create, and evaluate a musical system. You’ll need to be comfortable learning new languages and should enjoy working with physical hardware. It would be advantageous to have taken Sound and Music Computing.

You can take inspiration from some of our previous intelligent musical instruments that you can see here.

For an Honours/master project we would expect you to create a working prototype that includes IMPSy and enables interactive sound or music to be created. You would need to complete some type of formal evaluation. This project could also be the basis of a wider PhD project.

Please read information about joining the Sound, Music, and Creative Computing Lab before applying for this project.

How to Apply

To apply for a project in this area, contact Charles Martin.


  • your CV
  • your unofficial transcript (if you are an ANU student)
  • a brief statement (no more than 200 words) as plain text in your email explaining how you would approach this project

Make sure to specify the skills and accomplishments you have that would help you to complete this project. Have a look at the background papers below to gain some more knowledge about this project.

Background Papers and Resources:

You are on Aboriginal land.

The Australian National University acknowledges, celebrates and pays our respects to the Ngunnawal and Ngambri people of the Canberra region and to all First Nations Australians on whose traditional lands we meet and work, and whose cultures are among the oldest continuing cultures in human history.

arrow-left bars search times arrow-up