Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions documentation/components/bridges/symfony-postgresql-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,8 +143,15 @@ flow_postgresql:
rollback_file_name: "rollback.php" # Name of the rollback file in each version directory
all_or_nothing: false # Wrap all migrations in a single transaction
generate_rollback: true # Generate rollback files automatically
drop_if_exists: false # Emit IF EXISTS on diff-generated DROP statements
```

#### `drop_if_exists`

Set `drop_if_exists: true` to render every diff-generated `DROP` with `IF EXISTS` — useful for convergence migrations
that run against both fresh and legacy databases. Default `false`, so a missing object fails loudly (drift detection).
Override for a single run with the [`--drop-if-exists`](#generating-migrations-from-schema-diff) flag.

### Catalog Providers

Catalog providers define the target database schema. When you run `flow:migrations:diff`, the bundle compares
Expand Down Expand Up @@ -316,6 +323,7 @@ These commands are available when `migrations.enabled: true` for at least one co
| `--up` / `--down` | Migration direction (execute) |
| `--allow-empty-diff` | Don't fail when no changes detected (diff) |
| `--from-empty-schema` | Generate as if the database were empty (diff) |
| `--drop-if-exists` | Emit `IF EXISTS` on generated DROP statements (diff) |

## Migration Workflow

Expand Down Expand Up @@ -349,6 +357,13 @@ with the catalog, and `rollback.php` with the reverse operations (if `generate_r

Use `--from-empty-schema` to generate a migration as if the database were empty (useful for initial setup).

Use `--drop-if-exists` to emit `IF EXISTS` on every generated `DROP` for this run (a per-run override of the
[`drop_if_exists`](#drop_if_exists) config key):

```bash
php bin/console flow:migrations:diff --drop-if-exists
```

### Generating Blank Migrations

Create an empty migration for manual SQL (data migrations, custom operations):
Expand Down
37 changes: 37 additions & 0 deletions documentation/components/libs/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ execute it with the Client, and map results to objects.
| Add pagination to existing queries | [Query Modification](#query-modification) | `ext-pg_query` |
| Analyze query performance | [Query Plan Analysis](/documentation/components/libs/postgresql/client-explain.md) | `ext-pgsql`, `ext-pg_query` |
| Traverse or modify AST directly | [Advanced Features](#advanced-features) | `ext-pg_query` |
| Generate SQL from schema diffs | [Schema Diff](#schema-diff) | `ext-pg_query` |

## Requirements

Expand Down Expand Up @@ -687,6 +688,42 @@ foreach ($query->raw()->getStmts() as $stmt) {

---

## Schema Diff

Compare two catalogs and generate the SQL that turns the source into the target.

```php
<?php

use Flow\PostgreSql\Schema\Catalog;

use function Flow\PostgreSql\DSL\{catalog_comparator, schema, schema_column_integer, schema_table};

$current = new Catalog([schema('public', tables: [schema_table('users', [schema_column_integer('id', false)])])]);
$target = new Catalog([schema('public')]); // users removed

foreach (catalog_comparator()->compare($current, $target)->generate() as $sql) {
echo $sql->toSql() . "\n"; // DROP TABLE public.users CASCADE
}
```

### `dropIfExists`

By default, generated `DROP` statements have no `IF EXISTS`, so a missing object fails loudly (drift detection).
Set `dropIfExists: true` to render every diff-generated drop with `IF EXISTS` — useful for convergence migrations
that must run against both fresh and legacy databases:

```php
// On the comparator (applies to every compare):
catalog_comparator(dropIfExists: true)->compare($current, $target);

// Or per call (overrides the comparator's default):
catalog_comparator()->compare($current, $target, dropIfExists: true);
// -> DROP TABLE IF EXISTS public.users CASCADE
```

---

## Reference

### Exception Handling
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,13 @@ protected function configure(): void
->addOption('connection', 'c', InputOption::VALUE_OPTIONAL, 'The connection to use', null)
->addArgument('name', InputArgument::OPTIONAL, 'The name of the migration (e.g. \'add_categories\')')
->addOption('allow-empty-diff', null, InputOption::VALUE_NONE, 'Do not throw when no changes are detected')
->addOption('from-empty-schema', null, InputOption::VALUE_NONE, 'Generate as if the database were empty');
->addOption('from-empty-schema', null, InputOption::VALUE_NONE, 'Generate as if the database were empty')
->addOption(
'drop-if-exists',
null,
InputOption::VALUE_NONE,
'Emit IF EXISTS on generated DROP statements (overrides the migrations.drop_if_exists config for this run)',
);
}

protected function execute(InputInterface $input, OutputInterface $output): int
Expand All @@ -58,9 +64,10 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$name = $input->getArgument('name');
$allowEmpty = $input->getOption('allow-empty-diff') === true;
$fromEmpty = $input->getOption('from-empty-schema') === true;
$dropIfExists = $input->getOption('drop-if-exists') === true ? true : null;

try {
$version = $diffGenerator->generate($name, $allowEmpty, $fromEmpty);
$version = $diffGenerator->generate($name, $allowEmpty, $fromEmpty, $dropIfExists);
} catch (MigrationException $e) {
$io->warning($e->getMessage());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,10 @@ public function configure(DefinitionConfigurator $definition): void
->defaultTrue()
->info('Generate rollback files when creating migrations (default: true)')
->end()
->booleanNode('drop_if_exists')
->defaultFalse()
->info('Emit IF EXISTS on diff-generated DROP statements (default: false)')
->end()
->arrayNode('exclude')
->info('Schema objects excluded from migration diffing (e.g. tables created dynamically at runtime). Each entry defines exactly one matcher: schema, table, exact, starts_with, ends_with, pattern or policy_id.')
->arrayPrototype()
Expand Down Expand Up @@ -424,7 +428,7 @@ public function configure(DefinitionConfigurator $definition): void
}

/**
* @param array{connections: array<string, array{dsn: string, dbname: ?string, host: ?string, port: ?int, user: ?string, password: ?string, dbname_suffix: string, test_transaction_rollback: bool, context?: array<string, mixed>, telemetry?: array{service_id: string, clock_service_id: ?string, trace_queries: bool, trace_transactions: bool, collect_metrics: bool, log_queries: bool, max_query_length: int, include_parameters: bool, max_parameters: int, max_parameter_length: int}}>, messenger: array{enabled: bool, table_name: string, schema: string}, cache: array{pools?: array<string, array{connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, namespace: string, default_lifetime: int, marshaller_service_id: ?string, share_connection: bool}>}, session: array{enabled: bool, connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, lock_mode: string, ttl: ?int, share_connection: bool}, migrations: array{enabled: bool, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>}, catalog_providers: list<array{catalog_provider_id: ?string, catalog: ?array<string, mixed>}>} $config
* @param array{connections: array<string, array{dsn: string, dbname: ?string, host: ?string, port: ?int, user: ?string, password: ?string, dbname_suffix: string, test_transaction_rollback: bool, context?: array<string, mixed>, telemetry?: array{service_id: string, clock_service_id: ?string, trace_queries: bool, trace_transactions: bool, collect_metrics: bool, log_queries: bool, max_query_length: int, include_parameters: bool, max_parameters: int, max_parameter_length: int}}>, messenger: array{enabled: bool, table_name: string, schema: string}, cache: array{pools?: array<string, array{connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, namespace: string, default_lifetime: int, marshaller_service_id: ?string, share_connection: bool}>}, session: array{enabled: bool, connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, lock_mode: string, ttl: ?int, share_connection: bool}, migrations: array{enabled: bool, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, drop_if_exists: bool, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>}, catalog_providers: list<array{catalog_provider_id: ?string, catalog: ?array<string, mixed>}>} $config
*/
#[Override]
public function loadExtension(array $config, ContainerConfigurator $configurator, ContainerBuilder $container): void
Expand Down Expand Up @@ -666,7 +670,7 @@ private function registerMessenger(
}

/**
* @param array{enabled: bool, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>} $mc
* @param array{enabled: bool, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, drop_if_exists: bool, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>} $mc
*/
private function registerMigrations(string $name, array $mc, ContainerBuilder $container, bool $isFirst): void
{
Expand All @@ -684,6 +688,7 @@ private function registerMigrations(string $name, array $mc, ContainerBuilder $c
$mc['all_or_nothing'],
$mc['generate_rollback'],
$this->buildExclusionPolicy($mc['exclude'] ?? []),
$mc['drop_if_exists'],
]);
$configDef->setPublic(true);
$container->setDefinition("flow.postgresql.{$name}.migrations.configuration", $configDef);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,19 @@ public function test_generates_schema_migration(): void
static::assertStringContainsString('DROP TABLE', $rollbackContent);
static::assertStringContainsString('implements Rollback', $rollbackContent);
}

public function test_generates_schema_migration_with_drop_if_exists_flag(): void
{
$command = $this->context->command('flow.postgresql.command.diff');
$tester = new CommandTester($command);
$tester->execute(['name' => 'create_test_users_if_exists', '--drop-if-exists' => true]);

static::assertSame(Command::SUCCESS, $tester->getStatusCode());

$dirs = $this->context->migrationDirs('*_create_test_users_if_exists');
static::assertCount(1, $dirs);

$rollbackContent = $this->context->fileContent((string) $dirs[0] . '/rollback.php');
static::assertStringContainsString('DROP TABLE IF EXISTS', $rollbackContent);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -1151,6 +1151,7 @@ public function test_migration_configuration_values_propagated(): void
'namespace' => 'Custom\\Migrations',
'table_name' => 'custom_migrations_table',
'table_schema' => 'custom_schema',
'drop_if_exists' => true,
],
'catalog_providers' => [
['catalog_provider_id' => 'test.catalog_provider'],
Expand All @@ -1166,6 +1167,7 @@ public function test_migration_configuration_values_propagated(): void
static::assertSame('Custom\\Migrations', $configuration->migrationsNamespace);
static::assertSame('custom_migrations_table', $configuration->tableName);
static::assertSame('custom_schema', $configuration->tableSchema);
static::assertTrue($configuration->dropIfExists);
}

public function test_migration_generate_and_up_to_date_commands_registered(): void
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,22 @@ public function test_migrations_default_values(): void
static::assertSame('public', $config['migrations']['table_schema']);
static::assertFalse($config['migrations']['all_or_nothing']);
static::assertTrue($config['migrations']['generate_rollback']);
static::assertFalse($config['migrations']['drop_if_exists']);
}

public function test_migrations_drop_if_exists_can_be_enabled(): void
{
$config = $this->context->processConfig([
'connections' => [
'default' => ['dsn' => 'postgresql://user:pass@localhost:5432/db'],
],
'migrations' => [
'enabled' => true,
'drop_if_exists' => true,
],
]);

static::assertTrue($config['migrations']['drop_if_exists']);
}

public function test_migrations_exclude_defaults_to_empty(): void
Expand Down
8 changes: 7 additions & 1 deletion src/lib/postgresql/src/Flow/PostgreSql/DSL/schema.php
Original file line number Diff line number Diff line change
Expand Up @@ -1958,8 +1958,14 @@ function catalog_comparator(
?RenameStrategy $renameStrategy = null,
?ViewDependencyResolver $viewDependencyResolver = null,
?ExecutionOrderStrategy $tableOrderStrategy = null,
bool $dropIfExists = false,
): CatalogComparator {
return CatalogComparator::create($renameStrategy, $viewDependencyResolver, $tableOrderStrategy);
return CatalogComparator::create(
$renameStrategy,
$viewDependencyResolver,
$tableOrderStrategy,
dropIfExists: $dropIfExists,
);
}

#[DocumentationDSL(module: Module::PG_QUERY, type: DSLType::HELPER)]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ public function __construct(
public bool $allOrNothing = false,
public bool $generateRollback = true,
public ?ExclusionPolicy $exclusionPolicy = null,
public bool $dropIfExists = false,
) {}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,16 @@ public function __construct(
private bool $generateRollback = true,
) {}

public function generate(?string $name = null, bool $allowEmpty = false, bool $fromEmptySchema = false): Version
{
public function generate(
?string $name = null,
bool $allowEmpty = false,
bool $fromEmptySchema = false,
?bool $dropIfExists = null,
): Version {
$source = $fromEmptySchema ? new Catalog([]) : $this->sourceCatalog->get();
$target = $this->targetCatalog->get();

$diff = $this->comparator->compare($source, $target);
$diff = $this->comparator->compare($source, $target, $dropIfExists);

if ($diff->isEmpty() && !$allowEmpty) {
throw MigrationException::noChangesDetected();
Expand All @@ -41,7 +45,7 @@ public function generate(?string $name = null, bool $allowEmpty = false, bool $f
if ($this->generateRollback) {
$downSql = array_map(
static fn(Sql $query) => $query->toSql(),
$this->comparator->compare($target, $this->sourceCatalog->get())->generate(),
$this->comparator->compare($target, $this->sourceCatalog->get(), $dropIfExists)->generate(),
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ public function createDiffGenerator(MigrationGenerator $generator): DiffMigratio
return new DiffMigrationGenerator(
client_catalog_provider($this->configuration->client, exclusionPolicy: $exclusionPolicy),
$this->configuration->targetCatalogProvider,
catalog_comparator(),
catalog_comparator(dropIfExists: $this->configuration->dropIfExists),
$generator,
$this->configuration->generateRollback,
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ public function __construct(
private ExecutionOrderStrategy $materializedViewOrderStrategy = new MaterializedViewDependencyOrder(
new Parser(),
),
private bool $dropIfExists = false,
) {}

/**
Expand All @@ -44,6 +45,7 @@ public static function create(
?ExecutionOrderStrategy $tableOrderStrategy = null,
?ExecutionOrderStrategy $viewOrderStrategy = null,
?ExecutionOrderStrategy $materializedViewOrderStrategy = null,
bool $dropIfExists = false,
): self {
$renameStrategy ??= new GreedySimilarityRenameStrategy(new SimilarTextStrategy());
$constraintComparator = new ConstraintComparator();
Expand All @@ -64,16 +66,19 @@ public static function create(
$tableOrderStrategy,
$viewOrderStrategy,
$materializedViewOrderStrategy,
dropIfExists: $dropIfExists,
),
$viewDependencyResolver ?? new AstViewDependencyResolver(new Parser()),
$tableOrderStrategy,
$viewOrderStrategy,
$materializedViewOrderStrategy,
dropIfExists: $dropIfExists,
);
}

public function compare(Catalog $source, Catalog $target): CatalogDiff
public function compare(Catalog $source, Catalog $target, ?bool $dropIfExists = null): CatalogDiff
{
$effectiveDropIfExists = $dropIfExists ?? $this->dropIfExists;
$sourceNames = $source->names();
$targetNames = $target->names();

Expand All @@ -98,7 +103,11 @@ public function compare(Catalog $source, Catalog $target): CatalogDiff
continue;
}

$schemaDiff = $this->schemaComparator->compare($source->get($name), $target->get($name));
$schemaDiff = $this->schemaComparator->compare(
$source->get($name),
$target->get($name),
$effectiveDropIfExists,
);

if (!$schemaDiff->isEmpty()) {
$modifiedSchemas[] = $schemaDiff;
Expand All @@ -115,6 +124,7 @@ public function compare(Catalog $source, Catalog $target): CatalogDiff
$this->tableOrderStrategy,
$this->viewOrderStrategy,
$this->materializedViewOrderStrategy,
$effectiveDropIfExists,
);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public function __construct(
private ExecutionOrderStrategy $materializedViewOrderStrategy = new MaterializedViewDependencyOrder(
new Parser(),
),
private bool $dropIfExists = false,
) {}

/**
Expand All @@ -72,9 +73,10 @@ public function generate(): array
$dependentViews = $this->resolveDependentViews();

foreach ($dependentViews->toDrop as $dv) {
$sqls[] = $dv->view instanceof MaterializedView
$builder = $dv->view instanceof MaterializedView
? drop()->materializedView($dv->qualifiedName())
: drop()->view($dv->qualifiedName());
$sqls[] = $this->dropIfExists ? $builder->ifExists() : $builder;
}

foreach ($this->modifiedSchemas as $diff) {
Expand Down Expand Up @@ -109,7 +111,8 @@ public function generate(): array
}

foreach ($this->removedSchemas as $schema) {
$sqls[] = drop()->schema($schema->name)->cascade();
$builder = drop()->schema($schema->name)->cascade();
$sqls[] = $this->dropIfExists ? $builder->ifExists() : $builder;
}

return $sqls;
Expand Down
Loading
Loading