So far…

• sounds
• notes
• compositions
• connections

But how do we control any of it?

We’re missing the performer in the laptop music ensemble!

No performers: The Acousmonium (not in this class!)

There is always an interface!

Someone always has to press “play”.

We define an interface here as “software or hardware for controlling electronic music performance”

What kinds of interfaces are there?

Software Interfaces

interfaces can be based in software, e.g.,

• Graphical User Interface (GUI) elements in Pd
• Graphical interface design app (e.g., TouchOSC) running on tablet/phone (connected via network)
• Custom graphical UI created in Processing, p5.js, Unity, etc (talk to Pd via MIDI or network)
• Custom programming language to control Pd (over a local network): live coding systems

Hardware Interfaces

interfaces can be based in hardware as well:

• traditional human interface devices (HID): mouse, keyboard, joystick, dual-stick game controller
• non-traditional HID: webcam, microphone, laptop “drop sensor”
• commercial music interface devices: small piano keyboards, drum pads, fader/knob interfaces
• custom HID: microcontroller (e.g., MicroBit or Arduino) plus sensors
• separate interface computer: Raspberry Pi or smart phone with custom hardware/software communicating over a network connection

NIME community

“New Interfaces for Musical Expression”: https://nime.org

• …new kinds of musical instruments…
• …new kinds of music making…
• …and new kinds of music?

Software Interfaces

Pd GUI Objects

• Buttons: bang, toggle, message
• Sliders/Faders: vslider, hslider
• Selectors: Vradio, Hradio
• Documentation: comment, labels on other objects.

Try right-clicking sliders, bang or toggle to customise size, change colour, add labels.

Subpatches and Graph-on-Parent

To make sufficiently complicated Pd programs “neat”, you need to use subpatches and/or define your own objects.

• Subpatch: type pd and then the name for an object
• Own object: save a patch as a separate pd file, e.g., charlespatch.pd, then insert in another patch with it’s filename: charlespatch.

Use inlet, inlet~, outlet, outlet~ to get information in and out, and $0 to uniquely identify.

Use “graph-on-parent” (right-click properties) to make parts of a patch show up on the parent.

Cool Pd Interfaces are Good

It’s worth spending time on your Pd interfaces!

Clear and refined interfaces help others use your creativity support tools.

This is crucial for ensemble performance.

Graphical Interface in Processing

Processing is a good way to create a quick custom graphical interface.

Track the mouse, access a webcam, create game-like experiences, etc, with the power of Java.

Use the OscP5 library to send OSC messages.

N.B., this is highly related to, but not the same as p5.js.

Live Code Interfaces

Just as we used Processing to control Pd over OSC, we could just use any programming environment with real-time execution. E.g.,:

• terminal, use sendosc or just pdsend
• python, use python-osc
• any other live coding music system

There more detail on live coding later in this course.

Go do it in software

Make a a Pd software interface for a composition composition/synth.

Use graph-on-parent and subpatching to hide the DSP components, lets see the sliders!

Hardware Interfaces

Human Interface Devices in Pd

Keyboard is easy: key, keyup, keyname.

Mouse–not so easy:

• weird hacks overlaying hslider and an array of size 1
• xy.pd for a 2D mousing area abstraction

Used to be an external called hid but it’s very old.

(suggest looking at Processing for HID interactions)

Interfacing with audio…

• easy way to get some NOISE into your system, try interfacing with audio.
• pitch detection: fiddle
• onset detection: bonk
• try with voice, contact microphones, input from other performers, mix-down of the performance.

Camera

Pd can’t access a computer camera, but Processing can.

(I was super into webcam controllers ~2009)

better interfaces might use some computer vision techniques (own research or wait for week 12)

MIDI Controllers (2008)

MIDI Controllers (2022)

Commercial MIDI Controllers

I love MIDI controllers, but they are not always relevant to this class—mainly focussed on music production in a DAW (e.g., Ableton).

Keyboards: requires piano skills to be “good”, melodic music doesn’t always work well in LENS performances.

Remember that in computer music:

• buttons (binary discrete data) tend to less interesting
• sliders, faders, accelerometers, light sensors etc. which have rich, continuous data are more interesting

Connecting interfaces

• most commercial interfaces use MIDI over USB
• some modern devices use MIDI over Bluetooth
• phones/tablets connect using OSC over WiFi
• digital mixing desks and other equipment sometimes uses OSC over ethernet
• DIY option: serial over USB then translate to MIDI/OSC

Phones and OSC interfaces

A mobile device is a great controller:

• easy to hold
• amazing touch screen
• sensors (accelerometer and others)

Options:

• TouchOSC app
• DIY with MobMuPlat (works by running Pd on your phone with a special app to design a GUI)
• DIY with PdParty (just runs Pd patches on your phone)

NIME Microcontroller Workflow…

Best practices for DIY interfaces

The best way to control Pd with a microcontroller is to make it speak MIDI.

• Arduino MIDIUSB library
• Teensy USB_MIDI example

Alternatively, you can use a serial connection and translate to MIDI.

Interfaces and Ensembles

Interfaces for one are fun, but what about collaborating with interface data on a network (of some kind).

Collaborative interfaces require multiple musicians: just what you need for your LENS performance!

In Ensemble Feedback Instruments (Rosli et al. 2015), sound was passed around a group in a feedback network. Wild stuff.

Go do it in hardware

Make a hardware interface for your Pd patch. You can use either:

• keyboard or mouse movements in Pd
• sound input using fiddle~ and bonk~
• webcam tracking in processing + OSC

More on this later…

We will return to these topics in more depth!

• Live coding
• Hardware interfaces with the microbit
• Machine learning and AI in computer music interfaces