# Persistence

> A full-stack app keeps its data in a database that outlives the process; a schema says what that data may be, and migrations record how it changes

An app persists when what someone does in it is still there when they, or anyone
else, come back, across sessions, restarts and redeploys. That takes three
things: data kept somewhere that outlives the server process, a schema saying
what shape it may take, and a record of every change to that shape. This page
works through all three in [Django](https://www.djangoproject.com/), with the
equivalents in other stacks at the end.

## What survives a restart

A server process's memory lasts as long as the process, and on Fly the process
ends at every deploy, crash and auto-stop. Only the volume survives;
[running in production](https://comp.anu.edu.au/courses/comp4020-agentic-coding-studio/topics/running-in-production/#data-and-secrets) covers
why, and week 7's [what survives?](https://comp.anu.edu.au/courses/comp4020-agentic-coding-studio/lectures/week-7/#/what-survives) grid shows
it for every place state can live.

The trap is that memory takes the least code:

```ts
const messages: string[] = [];
```

It works in `pnpm dev` and passes any check that writes and reads back within
one run, and the next deploy empties it. After any change that stores something,
ask the agent where it put the state, and check the answer against the database.

## A schema says what's forbidden

A database schema is the tables, their columns and the constraints on them. Each
constraint is a rule the database enforces on every write, whichever code path
(or agent) did the writing. So the useful question about a column is what it
makes impossible.

```python
class Room(models.Model):
    name = models.CharField(max_length=40, unique=True)

class Message(models.Model):
    room = models.ForeignKey(Room, on_delete=models.CASCADE)
    body = models.CharField(max_length=500)
    created_at = models.DateTimeField(auto_now_add=True)
```

- **`unique=True`** means no two rooms share a name
- **no `null=True`** means every message has a room and a body. Django fields
  are `NOT NULL` unless you
  [say otherwise](https://docs.djangoproject.com/en/5.2/ref/models/fields/#null)
- **`ForeignKey`** means a message can't point at a room that doesn't exist, and
  `on_delete` says what happens to a room's messages when it's deleted:
  `CASCADE` deletes them, `PROTECT` refuses to delete the room

SQLite enforces less than most databases: it
[ignores the length](https://www.sqlite.org/datatype3.html) in `VARCHAR(500)`,
so `max_length` is checked by Django's forms and nothing else; a `CHECK`
constraint (`models.CheckConstraint` in Django) makes the database enforce it.

The schema is also where an app's design decisions show up. Who counts as a
person, what expires and who can see what are all columns and constraints before
they are features. An agent left to choose writes the most common schema for an
app like yours, and your app may not be the common one.

## Migrations

"Migrations are Django's way of propagating changes you make to your models
(adding a field, deleting a model, etc.) into your database schema"
([Django docs](https://docs.djangoproject.com/en/5.2/topics/migrations/)). Each
one is a file in the repo, and the database keeps a table of the ones it has
applied.

Add a `topic` field to `Message` and run `makemigrations`. Django compares the
models with the migrations so far and writes `0002_message_topic.py`:

```python
class Migration(migrations.Migration):
    dependencies = [("chat", "0001_initial")]

    operations = [
        migrations.AddField(
            model_name="message",
            name="topic",
            field=models.CharField(default="general", max_length=40),
        ),
    ]
```

`migrate` then applies every migration not yet in the `django_migrations` table,
in order. The same files run on your laptop and on Fly, so both databases end up
the same shape.

Three rules keep this working:

- **commit the model change and its migration together**, as the Django docs
  say, so every copy of the code has both
- **give a new non-null column a default**, since existing rows need a value.
  Without one, `makemigrations` stops and asks for a one-off default
- **never change the production database by hand**, and never edit a migration
  that has already run somewhere. Write a new one

### On Fly

With SQLite on a volume, migrations run when the server boots, since the volume
is attached only to that machine. Fly's `release_command` is the wrong place: it
[runs on a temporary machine](https://fly.io/docs/reference/configuration/) with
no volumes. In a Django `Dockerfile` that's
`CMD python manage.py migrate && gunicorn …`. The
[lecture demo](https://lecture-demos.fly.dev/) does the same thing from
TypeScript, calling Drizzle's `migrate()` at startup.

## The same ideas elsewhere

Every mainstream full-stack framework has a schema and a migration history. They
differ in whether migrations are generated from the schema or written by hand:

| framework             | schema lives in                               | new migration                                 | apply                                |
| --------------------- | --------------------------------------------- | --------------------------------------------- | ------------------------------------ |
| Django                | `models.py`                                   | `makemigrations` (generated)                  | `migrate`                            |
| Drizzle               | `schema.ts`                                   | `drizzle-kit generate` (generated)            | `migrate()` or `drizzle-kit migrate` |
| Rails (Active Record) | `db/schema.rb`, generated from the migrations | `bin/rails generate migration` (hand-written) | `bin/rails db:migrate`               |
| Phoenix (Ecto)        | schema modules                                | `mix ecto.gen.migration` (hand-written)       | `mix ecto.migrate`                   |

SQLite on one volume allows one writing machine. That suits a final project; an
app that has to run on several machines moves to Postgres, which is a decision
worth a [record](https://comp.anu.edu.au/courses/comp4020-agentic-coding-studio/topics/decision-records/).

## Further reading

- [The Twelve-Factor App: processes](https://12factor.net/processes) and
  [backing services](https://12factor.net/backing-services), on why the server
  keeps no state and the database lives outside the code
- [Fowler and Sadalage, "Evolutionary Database Design"](https://martinfowler.com/articles/evodb.html),
  on migrations as small, versioned steps committed with the code
- [Django: Migrations](https://docs.djangoproject.com/en/5.2/topics/migrations/)
  and the
  [model field reference](https://docs.djangoproject.com/en/5.2/ref/models/fields/)
- [Rails: Active Record Migrations](https://guides.rubyonrails.org/active_record_migrations.html)
- [Ecto.Migration](https://hexdocs.pm/ecto_sql/Ecto.Migration.html)
- [Drizzle: Migrations](https://orm.drizzle.team/docs/migrations)
- [SQLite: Datatypes](https://www.sqlite.org/datatype3.html), on what SQLite
  does and doesn't enforce
- [Fly.io: Volumes](https://fly.io/docs/volumes/overview/)