Skip to content

Consolidate 25 historical 2.x migrations into one end-of-2.8 schema#441

Merged
turegjorup merged 9 commits into
release/3.0.0from
feature/consolidate-migrations
May 27, 2026
Merged

Consolidate 25 historical 2.x migrations into one end-of-2.8 schema#441
turegjorup merged 9 commits into
release/3.0.0from
feature/consolidate-migrations

Conversation

@turegjorup

@turegjorup turegjorup commented May 6, 2026

Copy link
Copy Markdown
Contributor

Why now

A major version is the natural clean-cut point for retiring migration history. Consolidating into 3.0 means:

  • The 25 historical 2.x migrations are no longer replayed on every fresh install or CI run — boot time and test setup get measurably faster.
  • Operators upgrading 2.x → 3.0 do this once, with a documented step. There's no follow-up migration debt sitting in the repo.
  • The Template API surface change (removing the description query parameter, which dates back to 2.7) lands on the same major boundary, so 3.1 is internal cleanup only — no further user-visible API change required to finish the deprecation.

Doing this on a minor would mean introducing a coordinated cross-version operator step (rollup) and an API-filter trim mid-cycle, neither of which fit minor-version etiquette.

Summary

  • Squashes the 25 historical 2.x migrations (Version20220309093909 … Version20250828084617) into a single consolidated migration Version20260506215847.php representing the end-of-2.8 schema. Generated via doctrine:migrations:diff --formatted (not dump-schema, which silently omits FK constraints in doctrine/migrations 3.7).
  • Picks up feat: prep db identifiers for cross-platform (Postgres) portability #444's portability prep into the consolidated DDL: 15 <table>_changed_idx index names (Postgres-safe) and the backtick-quoted user table. The same shape is what 2.x → 2.8 → 3.0 upgraders end on via Version20260507120000, so fresh installs and rollups produce identical schemas.
  • Restores three previously removed Template entity properties (icon, resources, description) as deprecated, write-only fields with no getters/setters. The columns are kept in the consolidated 3.0 schema so fresh installs and 2.x → 3.0 upgraders end up with identical schemas, and Doctrine writes a value on every INSERT (the columns are NOT NULL with no DB default — without entity-side defaults, fixtures and app:templates:install fail).
  • Removes the previously 3.0-only Version20250815092648 (template column drop). The properties and the matching column-drop migration are scheduled for joint removal in 3.1 (TODO[3.1] markers throughout).
  • Adds a Template-specific template.search_filter (title only) so the deprecated description property is not re-exposed as a GetCollection query parameter. The shared entity.search_filter keeps description for other entities. #[ApiFilter] / #[ApiProperty] attributes were tried first but are silently ignored when the resource is YAML-configured.
  • Rewrites UPGRADE.md step 3: prerequisite check (latest 2.8.x with all migrations applied), then doctrine:migrations:rollup for upgraders (rewrites doctrine_migration_versions without re-running SQL). Fresh installs continue to use doctrine:migrations:migrate.
  • CHANGELOG entries under [Unreleased] for the consolidation and the deprecated-fields preservation.

Why diff and not dump-schema

doctrine:migrations:dump-schema produces a migration with zero FK constraints in this stack — only CREATE TABLE with inline indexes. migrations:diff against an empty DB produces the full schema including all 96 FK clauses.

Why entity restoration instead of an ignore-listener

An earlier iteration used a postGenerateSchemaTable listener to inject the deferred columns into the entity-side schema, with the entity left untouched. That kept schema:validate clean but broke INSERTs:

  • Fresh installs: fixtures couldn't INSERT templates (Field 'resources' doesn't have a default value).
  • Upgraders: would hit the same error the moment anyone ran app:templates:install, since rollup doesn't change schema.

The columns aren't truly dead — they're NOT NULL with no DB default. Restoring the entity properties (with their PHP-level defaults) makes Doctrine emit a value on every INSERT, fixing both populations uniformly.

Test plan

  • Fresh install end-to-end: drop test DB → create → migrations:migrateschema:validate exits 0
  • 2.x → 3.0 upgrade simulation: applied 25 latest-2.8.x migrations to fresh DB → replaced migration files with consolidated only → migrations:rollupmigrations:status reports 1 executed / 1 available / 0 unavailable → schema:validate exits 0
  • Full PHPUnit suite: 155 tests / 637 assertions green (post-release/3.0.0 sync brought in 6 new InstantBook tests)
  • PHPStan: no new errors
  • OpenAPI spec stays in sync after Template entity restoration (composer update-api-spec produces no diff)
  • Regenerated migration's DDL diff against the pre-feat: prep db identifiers for cross-platform (Postgres) portability #444 form is exactly the 15 index renames + 4 `user` quotings, nothing else
  • CI green on this push
  • Reviewer to verify the consolidated migration's DDL matches expectations against a real 2.8-final snapshot if available

Follow-up for 3.1

The three deprecated Template properties, the template.search_filter service, and the column-drop migration must land together in 3.1 — all carry TODO[3.1] markers. Removing one without the others would either re-introduce the schema mismatch (column without entity property) or the INSERT failure (entity expects the property gone but column still required).

🤖 Generated with Claude Code

@turegjorup turegjorup self-assigned this May 6, 2026
@turegjorup
turegjorup requested a review from tuj May 6, 2026 21:36
turegjorup and others added 3 commits May 7, 2026 00:07
Documents the 2.x → 3.0 consolidation operator flow before the actual
squash lands: rewrites step 3 to use `doctrine:migrations:rollup`
instead of `migrate` for upgraders (orphaned versions would otherwise
cause migrate to fail), notes the 2.7.x prerequisite, and keeps the
fresh-install path on `migrate` against the single consolidated file.

CHANGELOG also flags the temporary preservation of three `Template`
fields (icon, resources, description) as deprecated, write-only entity
properties so the consolidated migration represents end-of-2.7 schema
exactly. Fresh and upgraded installs stay symmetric for the eventual
3.1 drop, which removes both the entity properties and the columns
together.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-adds icon, resources, and description as deprecated, write-only
properties on the Template entity, matching their 2.7 mapping (string
NOT NULL + default '', JSON, string NOT NULL + default '') and PHP-level
defaults. No getters, no setters, no API exposure — they exist only so:

  1. The consolidated 3.0 migration creates columns matching the
     end-of-2.7 schema, leaving fresh installs and 2.x → 3.0 upgraders
     with identical schemas.
  2. Doctrine emits a value for each on every INSERT, since the columns
     are NOT NULL with no DB defaults — without the entity-side
     defaults, fixtures and `app:templates:install` would fail with
     "Field 'resources' doesn't have a default value".

TODO[3.1] markers on each property and a block comment call out paired
removal with the column-drop migration in 3.1.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… schema

Generated via `doctrine:migrations:diff --formatted` against an empty
test DB with the 3.0 entity metadata (which now carries icon, resources,
and description as deprecated properties matching 2.7). The diff command
is used because dump-schema in doctrine/migrations 3.7 silently omits
foreign key constraints; diff produces the full schema including all
96 FK clauses.

The 3.0-only `Version20250815092648` (template column drop) is removed
here as well — it's deferred to 3.1 along with the entity properties so
fresh and upgraded 3.0 installs stay schema-symmetric.

Verified end-to-end:
- Fresh install: drop DB → migrate → schema:validate exits 0
- Upgrade from 2.7: 25 latest-2.x applied → swap files → rollup → status
  shows 1 executed / 1 available / 0 unavailable; schema:validate exits 0
- Full PHPUnit suite: 143 tests / 607 assertions green

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@turegjorup
turegjorup force-pushed the feature/consolidate-migrations branch from 5a4735e to 1bd4a17 Compare May 6, 2026 22:08
@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown

API specification not up to date

The committed API specification files do not match the exported output.

Please run the following command, then commit and push the changes:

docker compose exec phpfpm composer update-api-spec

Restoring `Template::$description` re-activated the `entity.search_filter`'s
description partial-match on the Template GetCollection endpoint, which the
apispec workflow flagged as drift. A major version cut is the natural place
to retire API surface, so the parameter is removed in 3.0 — meaning the
eventual 3.1 column and entity-property removal is internal cleanup only,
no further API change.

Splits out a Template-specific `template.search_filter` (title only) and
points the GetCollection at it. Verified the regenerated OpenAPI spec
stays in sync.

API Platform's native attribute path (`#[ApiFilter]` / `#[ApiProperty]`)
doesn't work here: filter discovery via attributes only registers when the
resource itself is declared via attributes, but Template's resource is
YAML-configured (`config/api_platform/template.yaml`). The attribute is
silently ignored, leaving filter scoping as a config-side change.

TODO[3.1]: drop `template.search_filter` and switch GetCollection back to
the shared `entity.search_filter` once the deprecated entity property is
gone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@turegjorup
turegjorup marked this pull request as draft May 6, 2026 23:30
@turegjorup turegjorup changed the title feat: consolidate 25 historical 2.x migrations into one end-of-2.7 schema WIP: consolidate 25 historical 2.x migrations into one end-of-2.7 schema May 6, 2026
@turegjorup turegjorup changed the title WIP: consolidate 25 historical 2.x migrations into one end-of-2.7 schema WIP: consolidate 25 historical 2.x migrations into one end-of-2.8 schema May 27, 2026
turegjorup and others added 2 commits May 27, 2026 11:34
… entries

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…olidate-migrations

# Conflicts:
#	CHANGELOG.md
@turegjorup
turegjorup marked this pull request as ready for review May 27, 2026 09:37
turegjorup and others added 3 commits May 27, 2026 12:54
Picks up #444's portability fixes (15 changed_idx index renames and
backtick-quoted user table) into the consolidated end-of-2.8 schema so
fresh 3.0 installs and 2.x → 3.0 rollups land on the same DDL the
2.x → 2.8 → 3.0 upgrade path produces via Version20260507120000.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…olidate-migrations

# Conflicts:
#	CHANGELOG.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@turegjorup turegjorup changed the title WIP: consolidate 25 historical 2.x migrations into one end-of-2.8 schema Consolidate 25 historical 2.x migrations into one end-of-2.8 schema May 27, 2026
@turegjorup
turegjorup merged commit ad3bd91 into release/3.0.0 May 27, 2026
21 of 22 checks passed
@turegjorup
turegjorup deleted the feature/consolidate-migrations branch May 27, 2026 11:15
tuj pushed a commit to itk-dev/os2display-api-service that referenced this pull request Jun 2, 2026
- Rename 15 `changed_idx` indexes to `<table>_changed_idx` (Postgres
  scopes index names schema-wide; MariaDB scopes per-table — the shared
  name collides on Postgres).
- Quote `user` table identifier in entity metadata
  (`#[ORM\Table(name: '`user`')]`). Doctrine emits the platform-native
  quote on every reference (`` `user` `` on MariaDB, `"user"` on
  Postgres), so no table rename is needed.
- Add `validate-doctrine-schema-postgres` CI job that applies entity
  metadata to a Postgres 16 service container via
  `doctrine:schema:update --force --complete` and then runs
  `doctrine:schema:validate`. Gates against future entity-level
  Postgres regressions.

Lands on 2.7 so the consolidated 3.0 migration (os2display#441 / os2display#442) can emit
the portable shape from the start, keeping `migrations:rollup` honest
for 2.x → 3.0 upgraders.

Verified locally on MariaDB: migration applies cleanly,
`doctrine:schema:validate` in sync, full PHPUnit suite green
(133 tests / 516 assertions, identical to pristine release/2.7.0).
Verified on Postgres 16: `doctrine:schema:update --force --complete`
applies the metadata in 265 queries, `doctrine:schema:validate` in
sync. down/up cycle of the new migration also verified.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants