Lab 10: Layers Upon Layers

Outline

In this lab you will investigate layered composition using p5.Graphics layers and p5.FrameBuffer layers (for 3D, Shaders and GPU speed).

You will:

create graphics layers / frame buffers
render graphics to layers / frame buffers
compile the resulting graphics into an output layer

Fork and Clone Lab 10 from the GitLab Repository.

Graphics Layers

There are many many ways to think about, and to use tools, to create graphical output.

If you are familiar with image editing programs, you many be familiar with the concept of layers. You can create layers which have different content and that blend together in different ways.

You are already familiar with the concept of colours being defined using red, green, blue and alpha: with multiple layers of graphics, transparency is vital (if you have a layer with NO transparency it will obscure all lower layers).

The p5.js object underlying our graphics layer is the p5.Graphics object

Create Graphics Layer

To create and use a graphics layer you need a variable to store the graphics layer in, and a call to createGraphics().

You would usually create your graphics layers in function setup(), but you can run this command anytime (after preload()).


let graphicsLayer;

function setup() {
    createCanvas(windowWidth, windowHeight);
    graphicsLayer = createGraphics(windowWidth, windowHeight);
}

You may notice that createGraphics() and createCanvas() kind of look the same. createCanvas() creates a graphics layer that is also drawn directly to the webpage canvas element.

The syntax for createGraphics() is:

createGraphics(width, height, [renderer], [canvas])

Where:

Parameter	Description
width	The width of the graphics layer (in pixelDensity units)
height	the height of the graphics layer (in pixelDensity units)
renderer	P2D (default) or WEBGL
canvas	If supplied, the id of an existing html canvas element on the webpage

Note: graphics layers created through createGraphics may have different pixel densities (and thus render differently) to the base canvas. To avoid this you can set the pixel density via: pixelDensity(1);.

Draw to Graphics Layer

You can draw to a graphics layer created with createGraphics using ANY of the p5.js drawing commands. HOWEVER you have to tell p5.js to draw to your desired graphics layer as follows:

function draw() {
    // ... other code prior
    // DRAW A CIRCLE TO GRAPHICS LAYER
    graphicsLayer.fill(myColour); // set the fill colour for graphicsLayer drawing
    graphicsLayer.stroke(myStrokeColour); // set the stroke colour for graphicsLayer drawing
    graphicsLayer.strokeWeight(myStrokeWeight); // set the stroke weight for graphicsLayer drawing
    graphicsLayer.circle(x,y,r); // draw a circle at x,y with radius r to graphicsLayer
    // ... other code after
}

It is important to consider how you want your graphics to work with multiple layers — in terms of what the alpha channel is doing.

NOTE: if you do not call graphicsLayer.background(clr) then the background colour is (0,0,0,0) — it is transparent in the alpha channel for that graphics layer.

Compile Layers into Output Layer

We generally want to draw the graphics layer(s) we created to the canvas eventually.

To draw the graphics layer to the main canvas we call:

image(graphicsLayer,x,y,w,h);

If you have multiple layers:

image(graphicsLayer0,x,y,w,h);
image(graphicsLayer1,x,y,w,h);
image(graphicsLayer2,x,y,w,h); // etc.

You can choose to draw on the canvas after all of the graphics layers, or before all of the graphics layers (or event between graphics layers). Choose a strategy that works for you.

Blending

In general, non-transparent pixels will obscure any pixels below. Fully transparent pixels will allow all underlying pixels to be rendered. This is because the default blending is blendMode BLEND. Partially transparent pixels will be blended with the underlying layers.

You can create different effects by changing the blend mode between layers using the function blendMode().

I encourage you to experiment with blend modes again.

Possibilities

Daniel Shiffman in presenting createGraphics shows how you can create multiple copies of a graphics layer, you can tile these copies, you can tint them, you can translate() and rotate() them. See Additional Resources below.

FrameBuffer Layers for Shaders

WEBGL has a similar but different concept to graphics called FrameBuffers

p5.js FrameBuffers are like p5.Graphics objects in terms of drawing, layering and compositing, but are different in that they:

work on the GPU, not the CPU
are much faster than p5.Graphics
contain additional information, such as depth
only work in WEBGL mode

The benefits of speed, and the additional information available, make them a worthwhile addition to your toolkit of skills.

The p5.js object underlying our frameBuffer layer is the p5.FrameBuffer object

Creating FrameBuffers

Similar to graphics layers:

let frameBufferLayer;

function setup() {
    createCanvas(windowWidth, windowHeight, WEBGL); // must use WEBGL
    frameBufferLayer = createFrameBuffer(); // automatically defaults to canvas size
}

You can create multiple frameBuffers. While they can be sized directly, it is simplest to use the same size as your canvas.

You must create them after your canvas!

Drawing to FrameBuffers

This is a little different to p5.Graphics objects.

We tell the frameBuffer to begin drawing and to end drawing.

function draw() {
    // ... code prior
    frameBufferLayer.begin(); // any drawing instructions after this will draw onto the frameBufferLayer, not onto the canvas
    push();
    translate(mouseX - width/2, mouseY - height/2, -200);
    sphere(radius);
    pop();
    frameBufferLayer.end(); // resume drawing to the main canvas
    // ... code after
}

Compositing Frame Buffers

After you have drawn to a frame buffer you can use it as:

input to a shader

myShader.setUniform('tex0', frameBufferLayer.color);
myShader.setUniform('depth', frameBufferLayer.depth);

as a texture on a 3D surface

texture(frameBufferLayer);

as an image() — same as for p5.Graphics

image(frameBufferLayer,x,y,w,h); // remember (0,0) is the centre of the frame for WEBGL

Shady Ideas

Some shaders take a graphical image as input and modify the content of the graphical layer to create effects such as:

blurring
camera lens bloom
vignette
lens distortion
edge detection

If you investigate the filters available in many graphics editing programs, most (if not all, and more) of these are available as shaders.

In the p5.js frameBuffer tutorial Pagurek and Ferris explain how to use frameBuffering for video feedback using 2 frameBuffers which are swapped every frame.

Going deeper

The ability to access depth information enables you to create a variety of effects.

fog / haze that gradually obscures 3D elements that are further from the camera
depth of field: blurring objects that are closer or further than the desired focal plane — proving a photographic or cinematographic look
coloring / lighting based on depth

DO: Remember to commit your changes for this lab, and for Portfolio Item Two and push them up to Gitlab.