Michael McCullough
Semester 2, 2024
assignment 2 due this coming Monday 9pm!
now covered the coding topics.
have some more to learn about how to use p5
…and practice skills in art and design!
this week: sound and music computing!
next challenge: assignment 3!
why should you, the code artist, care about sound?
sound demands attention
sound communicates emotion
enhances realism & immersion
provides feedback during interaction
musical
abstract
skeuomorphic/sfx/diegetic
what sound to make?
when to start & stop?
how high (or low)?
how loud?
pitch: high or low (measured in Hz) ranging from 40Hz (low bass rumble) to 20kHz (high-pitched buzz)
amplitude: AKA volume or loudness, usually ranging from 0.0
(silence) to 1.0
(full volume)
duration: long or short, measured in seconds (it’s just time!)
timbre: often called the “colour” or “quality” of a sound (e.g. piano & saxophone can play the same pitch, but have different timbre)
what/when/how high/how loud
in computer music (and in p5.sound
), there are two ways to make a sound:
synthesis (mathematically calculating the signal)
sampling (playback of recorded sound)
Lots of options:
direct control through an input device (e.g. keyboard)
time-based
scene-based (e.g. connected to events in the sketch)
random
different pitches give us music
pitch can also communicate other things e.g. size, mood
slow changes in loudness—build tension, think verse vs chorus
quicker changes in loudness—accents in music
difference between loudest & softest sound is called the dynamic range—make use of it!
connect sound input to action on the screen
connect keyboard/mouse to sounds
connect camera/accelerometer to sounds
can we think about this as a new musical instrument?
synthesis in p5
!
sampling in p5
!
interactive music in p5
!
when we say frame rate, what are we talking about?
what’s the p5 draw()
loop frame rate? how would we find out?
what’s the normal sound frame rate?
44100Hz–96000Hz
44100 vs 60: that’s a huge difference (735x!)
p5 has a sound library: it’s called p5.sound
note: there’s no draw()
loop for sound to directly program frames of sound (it
would be too slow)
instead, p5.sound provides a bunch of sound “building blocks” (which are just objects!) which you need to create and modify.
You need to make sure the p5.sound
library is loaded in your index.html
sources produce sound: e.g., playing a sound file, “synthesising” a sine wave
processors process sounds: e.g., bass boost, reverb
p5.sound provides a
p5.Oscillator
which creates a
basic sound signal—here’s a simple example sketch
let osc;
function setup() {
osc = new p5.Oscillator();
osc.setType("sine"); // or "triangle", "sawtooth", "square"
osc.start();
}
function draw() {
osc.freq(1000*mouseY/height);
osc.amp(mouseX/width);
}
// assign p5.Oscillator object to variable
osc = new p5.Oscillator();
remember the key sonic interaction design questions:
.setType()
method
.start()
method
.freq()
method
.amp()
method
look at the reference for the full list of properties/methods
loadSound()
returns a
p5.SoundFile object useful for
playing pre-recorded sounds (e.g. mp3s, wav files, etc.)
let unilodgeSound;
function preload() {
unilodgeSound = loadSound("assets/unilodge.mp3");
}
function setup() {
unilodge.setVolume(0.5);
unilodge.play();
}
// assign p5.SoundFile object to variable
unilodgeSound = loadSound("assets/unilodge.mp3");
.play()
method, stop when file ends
(maybe).rate()
method
.setVolume()
method
note the subtle differences: the p5.SoundFile
object (sampling) has different
methods to the p5.Oscillator
object (synthesis)
preload()
functionwe’ve talked about setup()
and draw()
but in some of the examples (again, especially the sound/image ones) there’s a
preload()
function…
let’s check the reference
necessary because of the way the web browser works
where should unilodgeSound.play()
go?
function preload(){
// here?
}
function setup(){
// here?
}
function draw(){
// here?
}
function mousePressed(){
// here?
}
this course is about making your own interactive art works!
you’re not allowed to load and play background music that you didn’t create. (it’s boring and doesn’t help your work stand out).
you may only use sound files in your assignment 3 that you have recorded yourself.
the reference has the full list, but some highlights:
nope, sorry! we could have a whole course on computer music, but you know how to make some basic sounds in p5
let’s dig a bit deeper and look at making some interactive music.
Let’s look at a bit of synth theory…
We need to connect some events in time to an interactive input…
can sound be used as a material in art?
wait… isn’t that music?
how can we do this in p5
?
Éliane Radigue (b. 1932)
Éliane Radigue in her studio, Paris
c. 1970s
📷 Yves Arman
Musique Concrete Research Group (GRM)
François Bayle, Pierre Schaeffer and Bernard Parmegiani at GRM
1972
John Cage (1912-1992)
4’33”
1952
Piano score, three movements
John Cage (1912-1992)
Water Walk
1959
Various objects, microphone
watch: performed on US Television show I’ve got a secret (1960)
John Cage (1912-1992)
John Cage in the anechoic chamber
1951
anechoic chamber
Michael Asher (b. 1953)
A view of Michael Asher’s installation at Pomona College, looking out toward the street
1970
Pomona College of Art
Sound is spatial…
Let’s make some musique concrète.
Take sound files as a raw material…
and create an interactive music work.
p5.SoundFile
// assign p5.SoundFile object to variable
unilodgeSound = loadSound("assets/unilodge.mp3");
Playback control:
.pause()
stop playing (and resume later).resume()
un-pause.stop()
stop playing.Use the parameters for play
to create “notes”:
.play([startTime], [rate], [amp], [cueStart], [duration])
Change parameters:
.rate()
speed, pitch.jump()
change playback position.setLoop()
start/stop loopinguserStartAudio()
Browsers don’t like tabs to “autoplay” music. If you’re only using sound files, and .play()
ing them in setup
, some browsers won’t make any sound at all.
p5.sound
has a built-in trick to help, a function that starts audio that you can use after an interaction (e.g., clicking in the window):
function mousePressed() {
userStartAudio();
}
We can make “notes” out of soundfiles as well…
// in setup...
env = new p5.Envelope();
synthSample.amp(0);
synthSample.setLoop(true);
synthSample.play();
env.setInput(synthSample);
// somewhere else...
env.play();
p5.AudioIn
You can use p5.AudioIn
and p5.SoundRecorder
to access a microphone.
\\ setup
mic = new p5.AudioIn();
mic.start();
recorder = new p5.SoundRecorder();
recorder.setInput(mic);
soundFile = new p5.SoundFile();
\\ somewhere else...
recorder.record(soundFile);
recorder.stop();
Be very careful with this technique: feedback and unpredictable input are very annoying!
Chi-Hsia Lai and Charles Martin
2010
Computer video, graphics, vision, audio
p5.js
can do (almost) everything you could want with sound
To do more look at tone.js
or gibber
SYNTHESIZE ME: Sasha Frere-Jones on Éliane Radigue
Janet Cardiff and George Bures Miller, Night Walk for Edinburgh (2019)
Pierre Schaeffer: “etude aux chemins de fer”