Add schema migrations format

.env.template

@@ -0,0 +1,7 @@
+##### DBMATE Configuration
+# Configure the database connection
+DATABASE_URL=postgres://username:password@127.0.0.1:5432/database_name?sslmode=disable
+
+# Required path configuration
+DBMATE_MIGRATIONS_DIR=./migrations
+DBMATE_SCHEMA_FILE=./database.sql

.github/workflows/schema-validation.yml

@@ -0,0 +1,83 @@
+name: Schema Validation
+
+on:
+  pull_request:
+    paths:
+      - database.sql
+      - migrations/**
+      - .github/workflows/schema-validation.yml
+
+jobs:
+  validate-schema:
+    name: Validate Database Schema
+    runs-on: ubuntu-latest
+    env:
+      DATABASE_URL: postgres://postgres:password@localhost:5432/test_db?sslmode=disable
+      DBMATE_MIGRATIONS_DIR: ./migrations
+      DBMATE_SCHEMA_FILE: ./database.sql
+
+    services:
+      postgres:
+        image: postgres:15
+        env:
+          POSTGRES_PASSWORD: password
+          POSTGRES_USER: postgres
+          POSTGRES_DB: test_db
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        ports:
+          - 5432:5432
+
+    steps:
+      - name: Checkout sources
+        uses: actions/checkout@v4
+
+      - name: Install dbmate
+        run: |
+          curl -fsSL -o dbmate https://github.com/amacneil/dbmate/releases/latest/download/dbmate-linux-amd64
+          chmod +x dbmate
+          sudo mv dbmate /usr/local/bin/
+
+      - name: Validate schema is in sync with migrations
+        run: |
+          # Apply migrations, dumps schema
+          dbmate up
+
+          # Check if the schema file has changed
+          if git diff --exit-code database.sql; then
+            echo "✅ Schema validation passed - database.sql is in sync with migrations"
+          else
+            echo "::error::Schema file is out of sync with migrations!"
+            echo ""
+            echo "The database.sql file does not match what the migrations produce."
+            echo ""
+            echo "To fix this, run a postgres instance locally."
+            echo "Then apply the migrations to a clean database:"
+            echo "  dbmate up"
+            echo ""
+            echo "Then include the updated schema in your commit:"
+            echo "  git add database.sql && git commit -m 'Update schema'"
+            exit 1
+          fi
+
+      - name: Test that down migrations work
+        run: |
+          # Drop db from previous test
+          dbmate drop
+          # Apply migrations
+          dbmate up
+
+          for i in ./migrations/*
+          do
+            if test -f "$i"
+            then
+              # The command will fail if something is wrong
+              dbmate down
+            fi
+          done;
+
+          # Check if migrations can be run again
+          dbmate up

README.md

@@ -1,3 +1,44 @@
 # database-specs
 
-Contains the schema used for the Verifier Alliance database
+This repository contains the schema used for the Verifier Alliance database. The full schema is contained in `./database.sql`. This file is a generated dump from all applied schema migrations. The initial database schema can be found in `./migrations/20250717103432_database.sql`. It includes documentation about the schema and its tables.
+For a graphical representation of the schema, you can visit the [VerA docs](https://verifieralliance.org/docs/database-schema).
+
+## Migrations
+
+For keeping track of schema changes, we use a lightweight tool called [dbmate](https://github.com/amacneil/dbmate). Please follow the instructions on its GitHub repository for installation.
+
+### Prerequisites
+
+As a prerequisite for using dbmate, you should have a `.env` file configured with the database connection details.
+For this, copy the `.env.template` file to `.env` and replace the database connection string.
+
+### Running migrations
+
+For running all pending migrations against the database configured in `.env`, you can simply run:
+
+```bash
+dbmate migrate
+```
+
+### Adding a new migration
+
+Any migration added should be capable of updating the live VerA production and test database.
+This means that a migration should also transform data if necessary.
+
+For setting up a new migration, please use a fresh database, because dbmate will dump the current state of the database into `database.sql` after running the migration.
+For using a fresh database, simply configure an unused database name in the `.env` file.
+
+The process for adding a new migration is as follows:
+
+1. Create a new migration file: `dbmate new <migration_name>`
+2. Add the SQL needed for the schema change in the generated migration file (e.g., `migrations/20250717103432_add_new_table.sql`)
+   - The migration file should also include a `down` section to revert the changes if necessary.
+   - It should transform any data if necessary, e.g., for filling a new column with default values or copying data from one table to another.
+   - Ideally, the migration file should be idempotent, meaning it can be run multiple times without causing errors.
+3. Create a fresh database with the name configured in `.env`: `dbmate create`
+4. Run the migrations on the new database in order to generate the `database.sql` dump: `dbmate migrate`
+5. You can drop the database again: `dbmate drop`
+6. Both the migration file and the updated `database.sql` should be committed to the repository.
+
+Finally, any changes to the schema should be reflected in the `tests/`.
+Please update the tests if necessary, or add new tests to cover the changes made in the schema.