Skip to content

Commit abf95d5

Browse files
committed
Schema Skip/Upsert via pre-check + updatedAt gate on attribute/index
Re-migration needs the destination to tolerate existing schema resources. The database library stays minimal — migration owns the reconciliation decision since it has both source and destination metadata in hand. Per-resource behavior: - Database: pre-check `databases` metadata. If exists, hydrate sequence from the existing document and return. Both Skip and Upsert tolerate unconditionally — databases contain the entire tree of collections + rows, dropping would destroy everything. - Table: pre-check `database_{sequence}` metadata. Same pattern — tolerate unconditionally, hydrate sequence, never drop. Attribute and index reconciliation happens per-resource at the lower layer. - Attribute: pre-check `attributes` metadata by composite id ({databaseSeq}_{tableSeq}_{key}). Skip tolerates. Upsert compares source's updatedAt vs destination's — if source is strictly newer the column is dropped (via `dbForDatabases->deleteAttribute`) and recreated so the spec matches source. Column data is wiped on drop; rows will be repopulated by the row-level Upsert path that follows. - Index: same pattern against `indexes` metadata. Drop is cheap because indexes carry no user data — only rebuild time. Approach is pre-check rather than try/catch for control flow: migration is sequential single-writer, so the race condition justification for try/catch doesn't apply, and pre-check reads top-to-bottom with no exception-as-control-flow. Row-level dispatch (unchanged, already committed): OnDuplicate::Upsert → $dbForDatabases->upsertDocuments(...) OnDuplicate::Skip → $dbForDatabases->skipDuplicates(fn () => createDocuments(...)) OnDuplicate::Fail → plain createDocuments Both row-level primitives are existing library APIs — no database-library changes are required for this feature. Helpers: shouldTolerateSchemaDuplicate() // onDuplicate !== Fail sourceSpecIsNewer($src, $dst) // strtotime compare, false on empty
1 parent 74f166b commit abf95d5

1 file changed

Lines changed: 116 additions & 1 deletion

File tree

src/Migration/Destinations/Appwrite.php

Lines changed: 116 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,20 @@ protected function createDatabase(Database $resource): bool
422422
$createdAt = $this->normalizeDateTime($resource->getCreatedAt());
423423
$updatedAt = $this->normalizeDateTime($resource->getUpdatedAt(), $createdAt);
424424

425+
// Skip/Upsert: pre-check an existing database. Databases contain the
426+
// entire tree of collections + rows, so they are never destructively
427+
// reconciled — both modes simply hydrate the sequence from the
428+
// existing metadata document and return. Downstream resources
429+
// (tables/columns/rows) locate the existing underlying schema via the
430+
// hydrated sequence.
431+
if ($this->shouldTolerateSchemaDuplicate()) {
432+
$existing = $this->dbForProject->getDocument('databases', $resource->getId());
433+
if (!$existing->isEmpty()) {
434+
$resource->setSequence($existing->getSequence());
435+
return true;
436+
}
437+
}
438+
425439
$database = $this->dbForProject->createDocument('databases', new UtopiaDocument([
426440
'$id' => $resource->getId(),
427441
'name' => $resource->getDatabaseName(),
@@ -456,6 +470,32 @@ protected function createDatabase(Database $resource): bool
456470
return true;
457471
}
458472

473+
/**
474+
* Skip/Upsert both tolerate an existing schema resource on re-migration.
475+
* Upsert differs from Skip only for leaf resources (attributes, indexes)
476+
* where the updatedAt comparison gates a drop+recreate — containers
477+
* (databases, tables) are always tolerate-only because their data is
478+
* preserved.
479+
*/
480+
private function shouldTolerateSchemaDuplicate(): bool
481+
{
482+
return $this->onDuplicate !== OnDuplicate::Fail;
483+
}
484+
485+
/**
486+
* Upsert reconciliation: source's updatedAt strictly later than
487+
* destination's signals the spec changed since last sync — leaf resource
488+
* should be dropped + recreated. Equal or earlier: dest is up-to-date,
489+
* tolerate.
490+
*/
491+
private function sourceSpecIsNewer(string $sourceUpdatedAt, string $destUpdatedAt): bool
492+
{
493+
if ($sourceUpdatedAt === '' || $destUpdatedAt === '') {
494+
return false;
495+
}
496+
return \strtotime($sourceUpdatedAt) > \strtotime($destUpdatedAt);
497+
}
498+
459499
/**
460500
* @throws AuthorizationException
461501
* @throws DatabaseException
@@ -503,6 +543,21 @@ protected function createEntity(Table $resource): bool
503543
$dbForDatabases->create();
504544
}
505545

546+
// Skip/Upsert: pre-check an existing table. Like databases, tables
547+
// contain user data (rows), so both modes simply hydrate the sequence
548+
// from the existing metadata document. Attribute/index reconciliation
549+
// happens per-resource at a lower layer.
550+
if ($this->shouldTolerateSchemaDuplicate()) {
551+
$existing = $this->dbForProject->getDocument(
552+
'database_' . $database->getSequence(),
553+
$resource->getId()
554+
);
555+
if (!$existing->isEmpty()) {
556+
$resource->setSequence($existing->getSequence());
557+
return true;
558+
}
559+
}
560+
506561
$table = $this->dbForProject->createDocument('database_' . $database->getSequence(), new UtopiaDocument([
507562
'$id' => $resource->getId(),
508563
'databaseInternalId' => $database->getSequence(),
@@ -642,9 +697,38 @@ protected function createField(Column|Attribute $resource): bool
642697
$createdAt = $this->normalizeDateTime($resource->getCreatedAt());
643698
$updatedAt = $this->normalizeDateTime($resource->getUpdatedAt(), $createdAt);
644699
$dbForDatabases = ($this->getDatabasesDB)($database);
700+
701+
// Skip/Upsert: pre-check against `attributes` metadata. Skip tolerates
702+
// unconditionally; Upsert tolerates unless source's updatedAt is
703+
// strictly newer — in which case the column is dropped + recreated so
704+
// the spec matches source. Column data is wiped on drop; rows are
705+
// repopulated via row-level Upsert below.
706+
$attributeMetaId = $database->getSequence() . '_' . $table->getSequence() . '_' . $resource->getKey();
707+
if ($this->shouldTolerateSchemaDuplicate()) {
708+
$existingAttr = $this->dbForProject->getDocument('attributes', $attributeMetaId);
709+
if (!$existingAttr->isEmpty()) {
710+
if ($this->onDuplicate === OnDuplicate::Skip
711+
|| !$this->sourceSpecIsNewer($updatedAt, $existingAttr->getUpdatedAt())
712+
) {
713+
// Destination already up-to-date; hydrate caches and skip.
714+
$this->dbForProject->purgeCachedDocument('database_' . $database->getSequence(), $table->getId());
715+
$dbForDatabases->purgeCachedCollection('database_' . $database->getSequence() . '_collection_' . $table->getSequence());
716+
return true;
717+
}
718+
// Source spec newer: drop then recreate under the create flow.
719+
$dbForDatabases->deleteAttribute(
720+
'database_' . $database->getSequence() . '_collection_' . $table->getSequence(),
721+
$resource->getKey()
722+
);
723+
$this->dbForProject->deleteDocument('attributes', $attributeMetaId);
724+
$this->dbForProject->purgeCachedDocument('database_' . $database->getSequence(), $table->getId());
725+
$dbForDatabases->purgeCachedCollection('database_' . $database->getSequence() . '_collection_' . $table->getSequence());
726+
}
727+
}
728+
645729
try {
646730
$column = new UtopiaDocument([
647-
'$id' => ID::custom($database->getSequence() . '_' . $table->getSequence() . '_' . $resource->getKey()),
731+
'$id' => ID::custom($attributeMetaId),
648732
'key' => $resource->getKey(),
649733
'databaseInternalId' => $database->getSequence(),
650734
'databaseId' => $database->getId(),
@@ -920,6 +1004,37 @@ protected function createIndex(Index $resource): bool
9201004
);
9211005
}
9221006

1007+
// Skip/Upsert: pre-check against `indexes` metadata. Same rule as
1008+
// attributes — Skip tolerates unconditionally; Upsert tolerates unless
1009+
// source's updatedAt is strictly newer, in which case the index is
1010+
// dropped + recreated. Index drops are non-destructive (no row data
1011+
// lives in indexes) so rebuild cost is just the index build time.
1012+
$indexMetaId = $database->getSequence() . '_' . $table->getSequence() . '_' . $resource->getKey();
1013+
if ($this->shouldTolerateSchemaDuplicate()) {
1014+
$existingIdx = $this->dbForProject->getDocument('indexes', $indexMetaId);
1015+
if (!$existingIdx->isEmpty()) {
1016+
if ($this->onDuplicate === OnDuplicate::Skip
1017+
|| !$this->sourceSpecIsNewer($updatedAt, $existingIdx->getUpdatedAt())
1018+
) {
1019+
$this->dbForProject->purgeCachedDocument(
1020+
'database_' . $database->getSequence(),
1021+
$table->getId()
1022+
);
1023+
return true;
1024+
}
1025+
// Source spec newer: drop then recreate.
1026+
$dbForDatabases->deleteIndex(
1027+
'database_' . $database->getSequence() . '_collection_' . $table->getSequence(),
1028+
$resource->getKey()
1029+
);
1030+
$this->dbForProject->deleteDocument('indexes', $indexMetaId);
1031+
$this->dbForProject->purgeCachedDocument(
1032+
'database_' . $database->getSequence(),
1033+
$table->getId()
1034+
);
1035+
}
1036+
}
1037+
9231038
$index = $this->dbForProject->createDocument('indexes', $index);
9241039

9251040
try {

0 commit comments

Comments
 (0)