Lab 7: In Conversation

Outline

In this lab you will:

1. think a little bit about where the training data from these AI models comes from
2. use multiple charRNN models to generate multiple text outputs and produce a conversational AI sketch
3. Start your blog post

Intro

We’ve dived into the deep end of Artificial Intelligence, and we need to take a step back to look at how our AI system works.

Types of AI

There are multiple types of Artificial Intelligence… woah, back up: Type here can refer to expected capabilities [narrow vs broad], problem solving domain [goals or purpose], class of algorithm [techniques], or area of application [where are we using the AI and what are we using it for].

In this lab we are looking at Type as Techniques. Some of the main techniques of AI include:

1. Expert Systems
2. Search algorithms
3. Logic
4. Probabilistic Methods
5. Classifiers and statistical learning models
6. Artificial Neural Networks
7. Deep Learning
8. Generative Models with Transformers

Within each of these there are many individual approaches and algorithms. Within our “narrow AI” field, as computer scientists we need to select appropriate techniques to meet the goals of the system, use available information (input data sources), and available resources (compute power), and be a better approach than “traditional computer science” approaches.

CharRNN uses a Recurrent Neural Network. This is a type of Artificial Neural Network. Recurrent Neural Networks differ from other Neural Network architectures through including layers which “feed back” to previous layers. This feedback enables learning across different dimensions, including temporal (or sequential) learning. It also enables a generative capability.

Neural networks were developed as part of an early attempt to understand human cognition through creating a model of the neuron to understand how biological systems learn.

Machine Learning

Machines “learn” through finding patterns in data to make predictions or generate probable responses.

This implies a process of “learning” from data - and that is pretty much how it works.

The machine learning workflow process looks something like this:

1. Identify, locate, and collect data relevant to the task
2. Verify and clean the data
3. Prepare (pre-process) the data for training
4. Initialise a model (suitable for the relevant task)
5. Split data into train and test
6. “Train” the model using the training data set
7. “Test” the model using the test data set
8. Rinse and repeat (steps 1 through 7) until test validity is established
9. Deploy the model (use on new, unseen, data)
10. Monitor for efficacy (ready to complete cycle again)

There are many questions to ask here:

• what data is necessary for the task?
• where can we source the data?
• how do we know what is valid data?
• what do we lose through data preparation?
• what about “infrequent data” - how do we ensure representation?
• how is data biased? what can we do about that?

There are more questions to ask about data - and we haven’t even started on the other parts of the ML workflow.

Part 1: AIs and cooked output?

What do you know about the training data which was used for creating the charRNN models in this particular project (i.e. the subfolders in the models/ folder)?

If you look at the models/README.md file, it says:

This directory contains a small collection of pre-trained models for use with the ml5.charRNN(). Instructions for how to train your own model can be found in the training-charRNN repo. There are many complex and important ethical questions around voice, authorship, and consent when working with ML models that generate text trained off of a corpus of text. The following readings are suggested when working with the pre-trained models offered here.

• What Can Machine Learning Teach Us About Ourselves?, Interview with Emily Martinez, ml5.js Fellow 2020
• The Subtext of a Black Corpus, In conversation with ITP research fellows Nikita Huggins & Ayodamola Okunseinde by Ashley Lewis

There are no right answers here, because it’s complicated. And we don’t want to be reductive—all these authors are so much more than examples of AI bias or whatever. But it’s an important conversation to have as we use these (and other) AI models trained on real data from real people with real lived experiences, and it’s a good chance to at least think about the implications of this as we work in creative AI.

Part 2: Provocation

Your provocation for this week is “In conversation”. Using the outputs from at least 2 of the charRNN models you were introduced to last week, create a sketch which depicts a conversation.

Fork and clone the lab 7 template repo for this week.

Brainstorming interpretations

In the lab 7 template repo, you will find a planning.md markdown file. As a first step to this activity, write down some examples of what a conversation might look like. Write down as many ideas as you can in the next 10 minutes. Here are a few ideas you can extend on in your planning.md file:

• a text conversation (iMessage/WhatsApp/Messenger etc.)
• a reddit thread
• browser history (a conversation between search requests and results)
• song lyrics with multiple vocalists
• a play script
• a philosophical dialogue

Incorporating charRNN

Once you’ve come up with a few of your own ideas, think about how you can incorporate the outputs of the charRNN. For each of the interpretations of conversations in your planning.md file, write down how you can use the outputs of at least two of the charRNN models from last week.

How can you use the output from one model to influence the “response” of a different model? Hint: Think about how the “seed” sentence works from last week’s lab. What are some good conversation starters?

If you are unsure of how to implement an idea, write down what the idea is and why you find it challenging in the planning.md file. If, during the lab, you end up figuring out a way to implement it (even if its not exactly the solution you envisaged), write down your solution in the planning.md file.

If you’re struggling with exactly where to place the generated text, one function which you might find helpful is textWidth().

Part 3: Blog post submission

Since you will be submitting your creations for your blog post this week, we encourage you to generate 2-3 variations of your sketch. Think about how you can change the colour, composition, font and size to create variations in the way you communicate the conversation(s) you are depicting. If you haven’t already, take a look at the creative code blog deliverables page for more instructions. This page has all of the specifications for the blog post submissions.

If you didn’t finish the idea you wanted to implement during this lab, that’s all good. You can continue to work on it in your own time, as long as you submit 2-3 snapshots of the sketches you create before the next lab. Your sketches do not need to be polished – we just want to see that you

1. tried to explore the provocation and
2. tried to create several versions of your sketch.

As always, if you have any questions about the blog post, ask your instructor.

Before you leave class today, make sure you commit and push your work to git.

Summary

Congratulations! In this lab you:

1. explored where the training data from these AI models comes from
2. used multiple charRNN models to generate multiple text outputs to simulate a conversation
3. started your blog post about AIs in conversation