Welcome

--:-- recording — let's go

Week 6

Evidence of practice

what changed this week?

Debian voted on LLM usage in Debian

“responsible use of generative AI” won: disclosure of AI assistance is encouraged, not required

also lots of model releases

today’s beats

the transcript and the git log are the only account of the work that doesn’t rely on your memory

the “compound” step: write what you learned into the harness

CI runs the same checks on a machine that isn’t yours

demo: can it reconstruct fourteen years of my tooling from the git log alone?

then we leave it running and talk about what the work leaves behind

the practice leaves evidence

three records of the work

the transcript: every prompt, response and tool call

the git log: every diff and every commit

the CI runs: every pass and fail

the transcript is what agentic work adds, between the commits, on disk

(a running system that can examine itself is what programming languages call introspection)

the logs are just files

~/.claude/projects/<project>/<session>.jsonl, one JSON object per line (thousands in a long session); one record, trimmed to five fields:

{"type":"assistant",
 "timestamp":"2026-09-22T03:25:30.649Z",
 "cwd":"/home/ben/projects/comp4020",
 "gitBranch":"HEAD",
 "message":{"content":[
   {"type":"tool_use","name":"Bash",
    "input":{"command":"ls"}}]}}

querying the logs with jq

jq -r '.message.content[]?
       | select(.type=="tool_use") | .name' \
  <session-id>.jsonl | sort | uniq -c | sort -rn | head -4
    170 Bash
     30 Edit
      9 Read
      4 Write

(which tools did the agent call? one of my sessions, top four; your agent can write these for you)

writing the learnings back

compound engineering (Kieran Klaassen, Every): plan, work, review, compound — write the learnings back into persistent context, so the next cycle starts further along

you’ve been doing this in reflections/ and your CLAUDE.md since week 2

words; what do they even mean?

hoard things you know how to do

work a trick out once, keep the documented example, and it becomes an input to every later project

ref: Agentic Engineering Patterns

what have I worked out once and kept, in my public dotfiles history?

one line, thirteen months ago

  - use Australian English spelling
  - don't overuse lists (either bulleted/numbered or even
    just enumerating things in prose)
+ - do not ever use exclamation marks

a correction in the chat is gone next session; committed to claude/CLAUDE.md in August 2025, it is still in the file today

softened once on the way through — “don’t overuse lists or exclamation marks”

the log says what changed — does it say why?

the timeline falls out of the filenames and the dates alone

the reasons, where they survived, are in the commit messages

watch for the seam: where does the evidence stop and the story start?

and what does it reckon I’ll pick up next?

a reading of the past is cheap; a prediction can be checked

so we’ll have it commit the call, and check it next semester

the agent writes its own

CLAUDE.md is what you wrote; auto memory is notes Claude writes without being asked — your corrections, your preferences, facts about the project

~/.claude/projects/<project>/memory/, one markdown file per fact, with a MEMORY.md index read at the start of every session

on by default, per repo, machine-local, and yours to read, edit or delete: /memory

ref: How Claude remembers your project

a memory file, as Claude wrote it (trimmed)

---
name: uv-owns-the-venv
description: "Fix broken venvs with uv commands, never a manual …"
metadata:
  type: feedback
---

Let uv manage virtualenvs end to end. When a venv is broken … reach for
`uv sync --reinstall` …, not `rm -rf .venv`.

written after one correction; its one-line entry in MEMORY.md loads in every session in that project, and Claude reads the file when it looks relevant

a file in every session’s context

Cisco Talos got a poisoned MEMORY.md onto a machine through an npm postinstall hook

memory content was reaching the system prompt as though you had written it; Anthropic stopped that in v2.1.50

ref: Cisco Talos

the context cost of a rule

every rule you add is context you pay for in every later session, and it goes stale

it had silted up into documentation: superseded decisions, dated changelog entries, war stories

my commit message, cutting a CLAUDE.md from 669 lines to 250 (the rule that fell out went straight back in)

the chat lasts one session

the harness is loaded into every one

how much of this is true?

the number that circulates is 300–700% faster, sourced to a bracketed note citing nothing

Klaassen’s own claim is one hedged anecdote: two weeks, “maybe”, down to a few hours

faster at what, measured how, against what baseline?

Ben uses… a timer that ships the logs home

every session from every machine, synced into one sqlite table

> select host, count(*)
  from sessions group by host;
daysy |7625
weddle|8279

because the logs are just files

yes, and…

how does it get pushed further? weird ideas welcome

Web

continuous integration

a series of checks that runs on every push, on a machine that isn’t yours

GitHub Actions in our case, configured by .github/workflows/checks.yml in your repo; the idea is decades older

one push, one run

One push, one GitHub Actions run You run git push from your laptop. GitHub starts a fresh ubuntu-latest virtual machine with nothing of yours on it. The check job's steps run in order: checkout, mise-action to install the pinned node and pnpm, pnpm install with the frozen lockfile, pnpm check, pnpm check:evidence, a link check over dist, and a secret scan. The result comes back to the commit on GitHub as a tick or a cross, readable with gh run view, and the virtual machine is discarded. your laptop git push node_modules, your .env, your global tools: none of it leaves a fresh runner VM ubuntu-latest, nothing of yours on it push event actions/checkout mise-action (node, pnpm) pnpm install --frozen-lockfile pnpm check pnpm check:evidence linkinator ./dist trufflehog (secrets) ✓ or ✗ on the commit gh run view status then the VM is thrown away

which box does what?

the same checks at two distances

local: pnpm check in seconds, while the agent still holds the context that produced the mistake

the runner: the same command on a clean machine, which tests whether the code works anywhere other than the machine it was written on

if CI gives different results to local dev, then…

runner-only failures

✓ check   in 29s
✗ deploy  in 19s
    ✗ Run actions/configure-pages@v6
##[error] Get Pages site failed. Please verify that the
repository has Pages enabled and configured to build
using GitHub Actions

the demo repo’s first day, for real: every check green, and the deploy still failed

reading a red run

$ gh run view 37084179269
X deploy in 2m32s (ID 111090928068)
  ✓ Verify the deployed site is online
  X Verify the LiveView socket upgrades
$ gh run view 37084179269 --log-failed
the LiveView socket did not upgrade (HTTP 400).
If it is 403, the app thinks it is serving some
other host: check PHX_HOST and FLY_APP_NAME …
##[error]Process completed with exit code 1.

the failing step’s output gives the agent somewhere to look (lecture-demos, 3 October; trimmed)

always-red checks

week 3’s fourth property was trustworthy: you learn to scroll past a check that’s always red, and then the real failure goes unnoticed too

e.g. the crit template’s link check skips outbound links, because someone else’s rate limiter or outage would turn every build red (see the comment in checks.yml)

green when the sweep runs

the crit sweep starts fifteen minutes after your cutoff: pushed, live at its URL, reflection in the folder — and then it reads your checks

green: the full shipping mark

red, still running, or no checks: half

those fifteen minutes are settling time — push early enough that the run finishes

nothing is marked until you /ship

it flips the repo public, turns Pages on, triggers the deploy, then checks the URL actually serves

/preflight first — going public is the one irreversible thing you do in this course

(C6 is a retro: no prototype and no checks, so that half is just PROCESS.md in the repo by the cutoff)

over the break: fifteen minutes of fly

from week 7 CI deploys your repos on every push, and you never handle a deploy credential

but flyctl is how you read your running app’s logs and status: onboarding step 7, no account, no billing of your own

your tutor checks it works in c6 (no marks)

yes, and…

how does it get pushed further? weird ideas welcome

your assignment 2 repo is live

/comp4020:start clones it, like every other deliverable

the brief is final now — and the one thing that moved since last week is this morning’s lesson

PROCESS.md has a spine: what you decided a good course looks like, which of those decisions you encoded in CLAUDE.md or spec/, and which you left out

course you’d like to take you wish existed

update the course plugin

claude plugin marketplace update comp4020, then claude plugin update comp4020@comp4020, then restart Claude Code

do it before the break: assignment 2 ships on whatever copy you’re holding

yes, and…

how does it get pushed further? weird ideas welcome

did the steer show up in the result?

C6: the assignment 2 retro

no fresh provocation: the crit reflects on assignment 2

pick the specific change that made the course click and show it as a before/after — the breakthrough comes from the PROCESS.md you already submitted

the crit page has the spec

deadline radar

noon, Monday 21 September: assignment 2 — the first day back, so it’s closer than it looks

Monday/Wednesday, week 7: C6, with your PROCESS.md in the repo by the cutoff

the break: two weeks, and fifteen minutes of it is installing flyctl

tl;dr

the transcript, the git log and the CI runs are the only account of the work that isn’t recollection

write the correction into the harness rather than the chat, then prune it: every rule is context you pay for in every session

CI runs the same checks on a machine that isn’t yours — be green when the sweep runs, and /ship before the cutoff

ask your agent

ask until you can explain these back:

  • the compound step: what should this session’s corrections write back into your CLAUDE.md?
  • auto memory: open /memory and read what Claude has written about you
  • CI: which failures could only happen on the runner?
  • fly: install flyctl over the break, and confirm it runs

See you in the studio