Lab 10: Training Neural Networks

Outline

In this lab you will:

1. get introduced to ml5’s neural network
2. create your own data set and train a model
3. make a cool interactive [something] using machine learning

Part 1: Understanding Neural Networks

You might have heard of neural networks before. But what are they exactly? Neural networks were originally developed based on models of how neurons “learn” and respond selectively to different input signals. They were conceptually derived from the manner in which biological neurons in nervous systems signal to each other.

Artificial Neural Networks (ANNs) of the simplest types usually have three main layers:

• the input layer,
• one [or more] hidden layer[s], and
• an output layer.

Neural networks “learn” from input data. Learning is measured through a loss function: the lower the loss, the greater the accuracy [up to a point: if the loss is too low, the model may be overfitted]. ANNs have been used in image and speech recognition. We aren’t doing anything as complicated as that but we are going to train a neural network model and use it to classify unseen data.

We will be looking at ml5.js’s NeuralNetwork class, methods, and approaches.

Fork and clone this week’s template repo.

Oh my! Thats quite a bit of code. Its okay, we don’t expect you to understand everything. We are going to walk through the key concepts. The idea is to get a feel of creating your own data set and using that data set to generate sound, graphics, or other outputs.

Define the model variable: You might remember from previous labs where we used imported models that you have to define the machine learning model you are using. As mentioned we are using the ml5 NeuralNetwork. Copy the following into the set up function.

model = ml5.neuralNetwork(options);

Part 2: Neural Network Modelling Process

There are a number of steps to making a machine learning model:

1. collect data, create data, or load data from an existing dataset
2. define model parameters, and initialise the neural network
3. add data to the neural network
4. normalise your data
5. establish a test/train split
6. train the model
7. classification—use the model to classify some unseen data [aka prediction, inference]
8. do something with the results

Let’s start at the start and collect some data.

Part 3: Data Creation/Collection

Which Data?

Selecting data for Machine Learning is both a science and an art.

You need to clearly understand your problem domain, and analyse the systems involved the problem to understand how information is flowing through and influencing the system.

This is very, very complex: many issues with Machine Learning are the result of AI engineers or computer scientists NOT UNDERSTANDING the problem they are trying to address, and not investigating the systems to understand what influences the outcome of the system.

There may also be misunderstandings of the meaning/interpretation of the data. When data exhibits desired behaviours for your modelling task, scientists often place a standard interpretation on the data. You need to really look at your data to ensure you have a solid understanding of it.

Artistic Data

For our applications in creating art we can define the data to mean anything we want.

In this lab the data we are collecting are “notes” and their placement. We are going to plot down some letters creating a kind of map to train our model. The model is going to predict what letter you click on is and play the corresponding piano note with an oscillator.

Click the mouse; a C should appear. Click around the screen and mark some Cs down. Press the D key and click; a D should appear. Mark some Ds down in a cluster. Press the E key and do the same.

Once you have created your data its time to move onto the next step; training!

Part 4: Training Your Model

Press the t key to train the model. Make sure you don’t have caps lock on

You will notice a graph pop up. This graph shows you the number of ‘epoch’ and ‘loss’. Epoch refers to number of passes the entire training dataset the neural network has completed. The more epoch the greater the accuracy. Similar to when you are studying if you look over the same content over and over you are more likely to identify it accurately. Also similar to studying the more epochs the more time it takes to process. ‘loss’ is a function that quantifies the difference between the expected outcome and the outcome produced by the model. In other words it assess the accuracy of the model. You want the loss to go down over time.

Model is trained? Time to generate some sound!

Part 5: Using the Model

Now that we have trained our model, we want to generate some sound that corresponds to the data.

Add a call to the env.play() method inside your gotResults() function.

Now click around the screen. You should hear some musical notes created by the oscillator. If you click where you have placed a C you should hear a middle C note, if you click where you have placed a E you should hear an E note. And if you click in between the C and the E you should hear a note that sounds in between the C and the E notes.

If you’re using Chrome or Safari, you might need to put userStartAudio(); in the mousePressed() function if you aren’t hearing any sound.

It’s cool, but its only three notes—not that exciting. Let’s make some more notes!

In setup() there is a object called notes. What’s happening is oscillators create sound with frequencies. Every note has a corresponding frequency. For example F#3 (we say “F-sharp 3”) has a frequency of 184.9972Hz.

• use this table of piano frequencies in to complete a C major scale (C, D, E, F, G, A ,B)

• plot more data using the ‘C’, ‘D’, ‘E’, ‘F’, ‘G’, ‘A’ and ‘B’ notes

• train your model with the new data and play another C Major scale

C Major is a classic but let’s change it up. Look up some scales and choose one that you like. Find the corresponding frequencies for the notes and write it into the template.

Part 6: Saving the Model

At this point you might have gotten sick of clicking a billion times to plot data and wish you didn’t have to do it every time you reload the page. Well the good news is that you don’t have to—there is a way to save the data you collect and the model you have trained.

The ml5js neural network object has a .saveData() method (docs are here) which will save the data you’ve used to train your model as a json file.

For example, if your neural network object is in the model variable, then you can save the data to a json file with .saveData("mouse-notes.json") To load it back in, you want to make sure to move that json file back into your lab repo, otherwise when you .loadData() it won’t know where it is.

Remember that this just saves the data points, it won’t draw anything on the canvas. You have to write a few more lines of code to have it draw something too.

This video by Dan Shiffman goes into more detail and explains saving data and saving models.

Summary

In today’s lab you:

1. were introduced to ml5’s neural network
2. created your own data set and train a model
3. made a cool interactive drone synthesizer using machine learning

Remember that neural networks are super common in modern AI stuff, and you’ll definitely encounter them in all sorts of ways as you move through school and life. Some of the details are different in each case, but the basic “collect data/train/predict” pattern is the same, and having an understanding of that process is pretty useful. We’ll continue thinking about this stuff next week.

Before you leave class today, make sure you commit and push your work to git.