Welcome

--:-- recording — let's go

Week 10

Logs, harnesses and multi-user apps

what changed this week?

dynamic workflows came to Claude Managed Agents (Anthropic, 9 October): Claude Code’s workflows, moved to the API

a lead agent writes a program that runs many agents in phases, on Anthropic’s servers (“flows”, as the presenters called them at AI Engineer New York)

today’s beats

a demo: one symptom, two harnesses, the same logs

logs, and the harness around the model

multi-user apps: sessions, authorisation, two people at once

lecture-demos.fly.dev

demo: same model, same logs — what could be causing this?

the class noticed: [their call]. Read the app’s recent logs with mise exec -- flyctl logs -a lecture-demos --no-tail and tell us what could be causing it: your two likeliest explanations, and the evidence that would separate them. Change no code.

logging

a log

Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services.

(The Twelve-Factor App, “XI. Logs”)

the app writes to stdout; the platform collects it (on Fly, fly logs)

reading a request log

[287967] GET /rooms/1 200 8ms user=vQwIQ2r1
[01e7ba] card_created user=vQwIQ2r1 room=1 card=1
[01e7ba] POST /rooms/1?/add 200 11ms user=vQwIQ2r1

the tag is the request id: it ties each request to the events inside it

SvelteKit writes no request log of its own; the board’s is a handle hook, and each event is one line:

logEvent(event, "card_created", { room: roomId, card: added.id });

where the logs are

Where the logs are Five places, left to right along the path code takes. Your laptop: dev server output in the terminal that started it, readable if the agent started pnpm dev. GitHub Actions: each CI run, read with gh run view --log-failed. The Fly builder: the image build, printed by fly deploy. The Fly machine: the app's stdout and stderr, recent lines only, read with fly logs --no-tail. The browser: client-side errors in the DevTools console, which reach the agent only if you paste them in. git push deploy image requests your laptop dev server output, in the terminal that started it pnpm dev (agent-started) GitHub Actions each CI run: install, check, deploy gh run view --log-failed Fly builder the image build for each deploy fly deploy (its output) Fly machine the app's stdout and stderr, recent lines only fly logs --no-tail browser client-side errors in the console DevTools (paste it in) gold: a command the agent can run prints it

what to log

each event that matters: who, what, when, one line each

each error: enough context to reproduce it

never: secrets, tokens, or anything a user wouldn’t want in a log file

when something breaks: paste the lines around the failure into the agent, not a description of the symptom

further reading

The Twelve-Factor App, XI. Logs: logs as an event stream, and why the app shouldn’t manage log files

Fly.io, logging overview

SvelteKit docs, hooks

harnesses

a harness

Claude Code is the layer around the model that provides the tools and manages the context the model sees. This surrounding layer is what the term agentic harness refers to.

(Claude Code docs, “How Claude Code works”)

the course says “harness” for your CLAUDE.md and checks: the outer harness you build on top (Böckeler, 2026)

words; what do they even mean?

Claude Code and pi, before your first prompt

Claude Code and pi, before your first prompt Before the first prompt, a fresh Claude Code install sends about 26,000 tokens of system prompt and tool definitions, and offers the model 21 tools. A fresh pi install sends about 2,400 tokens and offers four tools: read, bash, edit and write. Claude Code 26.0k tokens Bash Read Edit Write NotebookEdit WebFetch WebSearch Task Skill SendMessage ListAgents TaskStop EnterWorktree ExitWorktree CronCreate CronDelete CronList ScheduleWakeup Workflow ReportFindings DesignSync pi 2.4k tokens read bash edit write

(fresh installs, same model: Claude Code 2.1.296, pi 1.1.0)

where they differ

featureClaude Codepi
permissionsasks, or a mode decidesno prompt per tool call; use a container
long-running commandsbackgrounded; output read as it arrivesbash waits until it exits; background it to a file, or use tmux
plan mode, subagentsbuilt inexample extensions, or pi -p from bash
MCPyesyes, since 1.0
the sessionJSONL; /branch forks a copya JSONL tree; /tree branches in place

how much the harness builds in

How much the harness builds in An axis from less built in to more built in. pi sits at the light end, with four tools and about 2,400 tokens before the first prompt; Codex CLI (a sandboxed shell with approval modes) and opencode (subagents, LSP and a plan mode) sit in the middle; Claude Code sits at the heavy end, with 21 tools and about 26,000 tokens. Extensions move pi to the right. The ends are measured on fresh installs; the middle placings are rough. less built in more built in pi 4 tools, 2.4k tokens Codex CLI sandboxed shell, approval modes opencode subagents, LSP, plan mode Claude Code 21 tools, 26k tokens extensions move pi right (the ends are measured on fresh installs; the middle placings are rough)

the harness’s own log

Claude Code: ~/.claude/projects/<dir>/<session>.jsonl

pi: ~/.pi/agent/sessions/--<dir>--/<time>_<id>.jsonl

plain JSONL, so any file tool reads them (your agent too). The tool calls:

# Claude Code
jq -r 'select(.type=="assistant") | .message.content[]?
  | select(.type=="tool_use") | .name' <session>.jsonl

# pi
jq -r 'select(.type=="message") | .message.content[]?
  | select(.type=="toolCall") | .name' <session>.jsonl

what carries over

CLAUDE.md: Claude Code, pi and opencode all read it (and Claude Code now falls back to AGENTS.md)

skills: the same SKILL.md format; pi looks in ~/.pi/agent/skills/ and ~/.agents/skills/

hooks, permission rules, plugins: Claude Code’s own; they don’t carry

course credits: any harness that speaks the Anthropic API (other tools); Claude Code is the supported path, the rest is going off piste

pi (using your course key)

install: mise use -g npm:@earendil-works/pi-coding-agent

configure: models.json and settings.json in ~/.pi/agent/, both in full on the course-credits page

you can’t run pi on a Claude sub

spawning agents

a subagent is a tool call: the harness starts a second loop with a fresh context and returns its result as the tool’s output (last week)

Claude Code: the Agent tool, built in

pi: no such tool in core; add one as an extension, or run pi -p from bash

subagents and workflows

Subagents and a dynamic workflow Left, subagents: a lead agent sends a task to each of three workers, and each worker's result returns to the lead's context, so the lead decides the next step turn by turn. Right, a dynamic workflow: the lead agent writes a script; the runtime runs a first phase of four agents and a second phase of two that check their work; intermediate results stay in script variables, and the final result returns to the lead. subagents lead agent worker worker worker every result lands in the lead's context the lead picks each next step dynamic workflow lead agent script agent agent agent agent checker checker result intermediate results stay in script variables the script picks each next step

(Claude Code docs, “When to use a workflow”)

dynamic workflows

A dynamic workflow is a JavaScript script that orchestrates many subagents at once. Claude writes the script for the task you describe, and a runtime executes it in the background while your session stays responsive.

(Claude Code docs)

start one: ultracode in a prompt; /workflows to watch it

keep one: saved to .claude/workflows/, it runs as /<name>

a workflow run

A dynamic workflow run Two lanes over time. In your session, you prompt with ultracode, the model writes a script, and you keep working. In the workflow runtime, one agent lists the route files, then four agents audit one file each, and the script filters their results. The result returns to your session. You can watch the run with /workflows, and the saved script in .claude/workflows/ runs again as /audit-routes. your session workflow runtime ultracode … model writes the script you keep working list route files audit one file audit one file audit one file audit one file filter result /workflows shows each agent as it runs saved to .claude/workflows/, it runs again as /audit-routes

a workflow script

export const meta = { name: "audit-routes", description: "…" };
const found = await agent("List every .ts file under src/routes/.", {
  schema: FILES,
});

const audits = await pipeline(found.files, (file) =>
  agent(`Audit ${file} for missing auth checks.`, { label: file }),
);
return audits.filter(Boolean);

the script only starts agents and combines what they return; each agent() call is a subagent with its own tools (docs)

orchestration in Claude Code and pi

featureClaude Codepi
subagentsAgent tool, built inexample extension: each child is a separate pi process
workflowsWorkflow tool and its runtime, built innot in core; community extensions run Claude Code’s scripts
scripts over toolsonly through agentscodemode runs JavaScript over tools, but not over chat models
concurrency, resumeAnthropic’s defaults: 16 at once, 1,000 per runwhatever the extension does

evals

an eval runs the same tasks through each harness and grades the results; public benchmarks (Terminal-Bench) use their tasks, not yours

replay your history: take a few past tasks from your git log, reset to the parent commit, give each harness the same prompt, grade with the checks you already have

start from failures: each time an agent gets something wrong, save the prompt and the check that caught it

(Anthropic, demystifying evals for AI agents)

Ben uses… vibes

mostly no evals: Ben judges a harness or model by how it goes on real work

further reading

Claude Code docs, how Claude Code works: the loop, the tools, the context window and where sessions are stored

Zechner (2025): why pi is four tools and a short prompt (some of it, like no MCP, has since changed)

pi docs: configuration, sessions and the session format

Böckeler (2026), harness engineering: the harness you build around a coding agent

Claude Code docs, dynamic workflows: subagents, agent teams and workflows compared, and the script API

Ben uses… a weekly retrospective

an agent reads the week’s session logs and proposes a rule for each correction Ben repeated; one from 4 October:

A global CLAUDE.md line: to hand Ben text he will send or paste, … pipe it to clipboard-copy

course usage so far

strproxy logs every request your harness makes (we last looked in week 4)

(totals and anonymous rankings — no names anywhere)

yes, and…

how does it get pushed further? weird ideas welcome

Web

multi-user, in SvelteKit

the worked instance today is the board in lecture-demos: SvelteKit, with better-auth for sign-in and Drizzle over SQLite

Django (week 8) and Phoenix (week 9) come with these parts built in; here you choose them, as your agent will for your final project

a multi-user app

two or more people, each in their own browser, act on shared state, and the app tells them apart

(the final project’s wording)

markers test it: two Chrome profiles, so no shared cookies or storage

three things follow: telling people apart, deciding what each may do, and handling two of them at once

telling people apart

sign-in sets a cookie holding a token (MDN):

__Secure-better-auth.session_token=EAzs…; HttpOnly; SameSite=Lax

each request, a handle hook looks the token up in the session table:

const session = await auth.api.getSession({
  headers: event.request.headers,
});
if (session) event.locals.user = session.user;

the session is a database row, so it survives a restart

a session, request by request

sign in, then every request after Ada's browser to server: POST /login?/signIn email, password. server to database: check the password insert session row. server to Ada's browser: Set-Cookie: …session_token=EAzs… HttpOnly; SameSite=Lax. Ada's browser to server: GET /rooms/1 Cookie: …session_token=EAzs…. server to database: look up the token → user Ada. server to Ada's browser: 200, the room as Ada sees it . Ada's browser server database POST /login?/signIn email, password check the password insert session row Set-Cookie: …session_token=EAzs… HttpOnly; SameSite=Lax GET /rooms/1 Cookie: …session_token=EAzs… look up the token → user Ada 200, the room as Ada sees it

authorisation

authentication says who; authorisation says what they may change, checked on the server for every action

const { changes } = db
  .delete(card)
  .where(and(eq(card.id, cardId), eq(card.userId, me.id)))
  .run();
if (changes === 0) error(404, "No such card");

Bo, deleting Ada’s card:

[2c9315] POST /rooms/1?/delete 404 3ms user=IES9bBIp

authentication, then authorisation

who is asking, and may they? Bo's browser to server: POST /rooms/1?/delete card=1, Cookie: Bo's token. server to database: authentication: token → user Bo. server to database: authorisation: DELETE card 1 WHERE user_id = Bo → 0 rows. server to Bo's browser: 404 No such card . Bo's browser server database POST /rooms/1?/delete card=1, Cookie: Bo's token authentication: token → user Bo authorisation: DELETE card 1 WHERE user_id = Bo → 0 rows 404 No such card

two people at once

timeAda’s browserBo’s browser
11:02:01opens card 1: “buy milk”
11:02:05opens card 1: “buy milk”
11:02:20saves “buy oat milk”
11:02:31saves “buy milk and bread”

card 1 now reads “buy milk and bread”: Ada’s edit is gone (a lost update), and the log shows two clean saves

[37a745] card_edited user=vQwIQ2r1 room=1 card=1
[e029e7] card_edited user=IES9bBIp room=1 card=1

race conditions

what should happen when two people save the same thing depends on what your app is for: last write wins, a version check, a lock, or merging the edits

talk it through with your agent, then have it write a test where two users save at once

further reading

MDN, Using HTTP cookies and the OWASP Session Management Cheat Sheet: sessions, and how they go wrong

better-auth, session management: the cookie, the table and expiry

SvelteKit docs, form actions: the board’s writes, and the CSRF check on every form POST

SQLite, transactions: one writer, and what a transaction does and doesn’t cover

yes, and…

how does it get pushed further? weird ideas welcome

next week: a guest lecture

Tom Griffiths, one of your tutors, spent 15 years as a product manager in tech companies small and not so small

Claude Code in industry: how it’s really used, and how to thrive when you walk into a job where everyone has it

no lecture in week 12

Thurs 29 Oct, 11am: an (optional) panel instead, Scientific Research in the Age of Gen AI, in Hanna Neumann 1.33

(nothing to do with this course’s assessment; its focus is academic research, not agentic coding)

register here only if you’re going: the room holds about 70

Fri 30 Oct, 1pm: a final-project drop-in, in our lecture room (Fulton Muir 2.02)

how A3’s requirements are tested

multi-user: two Chrome profiles (incognito counts), so no shared cookies

real-time: both on screen at once, one per marking viewport

persists: back the next day, after your machine has restarted

(see how it’s marked)

C10: fly by instruments

make your final project tell people apart, decide on the server what each may see and change, and write down why

at the crit two of your pod use it from the URL alone, in separate Chrome profiles

you don’t touch it: you narrate from your logs who did what

deadline radar

your week 11 session: C10 live two hours before it, with reflections/crit-10.md in the repo

noon Monday 9 November: the final project is due

12–2pm Wednesday 11 November: the final project showcase in the Birch building (optional)

tl;dr

logs help only when a command the agent can run prints them

a harness is everything around the model, spawning other agents included; the same model works differently inside Claude Code and pi

a multi-user app tells people apart with a session, checks every action on the server, and decides what happens when two people save at once

ask your agent

ask until you can explain these back:

  • logging: what should my app log for each user action, without secrets or personal data?
  • harnesses: what’s in your system prompt and tools, and what would differ in pi?
  • sessions: how does my stack tell two people apart, after a restart too?
  • conflicts: what happens when two people save the same thing at once?

See you in the studio