fix(cli): skip pg_dump in db pull when using pg-delta diff engine (#5255)

## What

Switches `supabase db pull --diff-engine pg-delta` to drive its initial
pull from pg-delta's catalog diff instead of `pg_dump`, and lands the
supporting plumbing (remote TLS handling, debug bundles, and a local
pg-delta dev workflow) needed to make that path reliable against Cloud.

Closes CLI-1472.

## Why

The legacy initial-pull pipeline is `pg_dump` → restore into a local
shadow → diff. That round-trip silently destroys ownership for any
object the local `postgres` role can't assume: `pg_dump` emits `ALTER
... OWNER TO supabase_admin` (etc.), the restore runs as a non-superuser
so the owner is dropped, and every restored object then reports `owner =
postgres`. pg-delta's supabase integration filter drops platform-managed
objects by `owner ∈ SUPABASE_SYSTEM_ROLES`, so once the owner signal is
gone those objects leak into the user's migration file and break
`supabase db reset` (CLI-1469 FDW ACLs, CLI-1470 Wasm wrappers).

pg-delta already speaks `pg_catalog` directly via `extractCatalog`,
preserving ownership, so diffing the remote catalog against an empty
shadow produces a clean initial migration without `pg_dump` in the loop.

## What changed

### Initial-pull routing
- `db pull` no longer runs `dumpRemoteSchema` on the initial pull when
the pg-delta diff engine is selected; the shadow diff is the sole
producer of the migration file (`internal/db/pull/pull.go`,
`cmd/db.go`).
- `shouldUseDeclarativePgDeltaPull` makes explicit `--diff-engine
pg-delta` keep the migration-file workflow even when
`[experimental.pgdelta] enabled = true` would otherwise default to the
declarative export path.
- New `swallowInitialInSync` / `ensureMigrationWritten` helpers
distinguish "pg-delta produced no statements" from "pg_dump already
wrote content", so an empty pg-delta result still surfaces `No schema
changes found` correctly.

### Diff plumbing
- `DiffDatabase` now returns a `DatabaseDiff` struct (`SQL` plus an
optional debug capture) rather than a bare string; callers in `diff.go`
and `pull.go` updated accordingly.
- Removed the now-unused `diffWithStream` and its golangci exclusion.
- The diff template (`templates/pgdelta.ts`) sets
`skipDefaultPrivilegeSubtraction: true` and emits structured diagnostics
under `PGDELTA_DEBUG`.

### Remote TLS handling for pg-delta
- New `internal/gen/types/pgdelta_conn.go`: `PreparePgDeltaPostgresRef`
writes the CA bundle into the workspace, forces `sslmode=verify-ca` +
`sslrootcert`, and always uses the embedded bundle for Supabase-hosted
hosts even when the SSL probe is skipped (e.g. `--debug`). Source/target
now use distinct `PGDELTA_SOURCE_SSLROOTCERT` /
`PGDELTA_TARGET_SSLROOTCERT` env vars. `pgdelta.go` and
`pgcache/cache.go` refactored onto this helper.

### Debug bundles for empty pulls
- New `PGDELTA_DEBUG` flag (separate from `--debug`; keeps SSL on). When
a pg-delta pull returns zero statements, the CLI writes a bundle under
`supabase/.temp/pgdelta/debug/<timestamp>/`: `source-catalog.json`,
`target-catalog.json`, `pgdelta-stderr.txt`, `connection.txt`
(redacted), and `error.txt`, plus a per-schema catalog object summary to
stderr (`internal/db/diff/pgdelta_debug.go`,
`internal/db/pull/pgdelta_pull_debug.go`).
- `DiffPgDeltaRefDetailed` surfaces edge-runtime stderr;
`declarative.DebugBundle` gains inline-catalog, stderr, and
connection-info fields.

### Local pg-delta dev workflow
- `PGDELTA_NPM_REGISTRY` (`pkg/config/pgdelta_local.go`,
`internal/utils/pgdelta_local.go`) routes Deno's
`npm:@supabase/pg-delta` resolution through a local Verdaccio registry.
The edge-runtime helper gained composable `WithExtraFile` /
`WithExtraEnv` options to drop a scoped `.npmrc` and forward
`NPM_CONFIG_REGISTRY`; all pg-delta edge-runtime calls now pass
`PgDeltaNpmRegistryOption()`. Documented in `CONTRIBUTING.md`.

### Version, build, and docs
- `DefaultPgDeltaNpmVersion` bumped to `1.0.0-alpha.27`.
- `apps/cli/package.json` adds a `build:go-sidecar` step that copies the
Go binary into `dist/`.
- `docs/supabase/db/pull.md` documents the new initial-pull behavior,
the experimental declarative default, the direct-connection
recommendation for `--db-url`, and the `PGDELTA_DEBUG` workflow.

## Reviewer notes
- The FDW/server ACL and `CREATE FOREIGN DATA WRAPPER` suppression rules
in `@supabase/pg-delta` become defense-in-depth on this path rather than
load-bearing.
- Scope is schema-only / migration generation: `supabase db dump` and
the declarative export path still use `pg_dump` / the existing flows.

https://claude.ai/code/session_016o7U7hiut6dkkhfbSJgKv4

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Julien Goux <hi@jgoux.dev>

Author: 3 people

apps/cli-go/.golangci.yml

@@ -16,7 +16,7 @@ linters:
   exclusions:
     generated: lax
     rules:
-      - text: 'ST1003:'
+      - text: "ST1003:"
         linters:
           - staticcheck
       - path: _test\.go
@@ -27,10 +27,7 @@ linters:
         linters:
           - gosec
         text: G117
-      - path: internal/db/diff/diff\.go
-        linters:
-          - unused
-        text: diffWithStream is unused
+
     presets:
       - comments
       - common-false-positives

apps/cli-go/CONTRIBUTING.md

@@ -41,3 +41,52 @@ go test ./... -race -v -count=1 -failfast
 ## API client
 
 The Supabase API client is generated from OpenAPI spec. See [our guide](api/README.md) for updating the client and types.
+
+## Testing local pg-delta builds
+
+To exercise unpublished `@supabase/pg-delta` changes inside CLI edge-runtime scripts (`db pull`, `db diff`, `db push`, etc.), publish a local build via Verdaccio in [pg-toolbelt](https://github.com/supabase/pg-toolbelt) and point the CLI at that registry.
+
+### 1. Start Verdaccio (pg-toolbelt)
+
+```sh
+cd pg-toolbelt
+bun run verdaccio:start
+```
+
+Verdaccio listens on `http://localhost:4873`. `@supabase/*` packages you publish locally are served from local storage; other `@supabase/*` dependencies (for example `@supabase/pg-topo`) are proxied to npmjs.
+
+### 2. Publish a local pg-delta build
+
+After changing `packages/pg-delta`:
+
+```sh
+bun run pg-delta:publish-local \
+  --write-version-to=/path/to/test-project/supabase/.temp/pgdelta-version
+```
+
+This publishes a fresh `0.0.0-local.<timestamp>` version and restores `package.json` afterward. The version file tells the CLI which npm version to request (`EffectivePgDeltaNpmVersion`).
+
+Re-run whenever you change pg-delta source.
+
+### 3. Run the CLI against the local registry
+
+Set `PGDELTA_NPM_REGISTRY` to a URL reachable **from inside the edge-runtime Docker container**:
+
+```sh
+# Docker Desktop (macOS / Windows)
+export PGDELTA_NPM_REGISTRY=http://host.docker.internal:4873
+
+# Linux (Docker 20.10+)
+export PGDELTA_NPM_REGISTRY=http://host.docker.internal:4873
+# or: export PGDELTA_NPM_REGISTRY=http://172.17.0.1:4873
+```
+
+Then run any pg-delta-backed command, for example:
+
+```sh
+supabase db pull --db-url "$DATABASE_URL" --diff-engine pg-delta
+```
+
+When set, the CLI injects a scoped `.npmrc` and forwards `NPM_CONFIG_REGISTRY` into the edge-runtime container (`PgDeltaNpmRegistryOption` in `internal/utils/pgdelta_local.go`).
+
+Unset `PGDELTA_NPM_REGISTRY` to return to the npmjs version pinned in config / `supabase/.temp/pgdelta-version`.

apps/cli-go/cmd/db.go

@@ -181,7 +181,7 @@ var (
 			if usePgDeltaDiff {
 				pullDiffer = diff.DiffPgDelta
 			}
-			useDeclarativePgDelta := shouldUsePgDelta()
+			useDeclarativePgDelta := shouldUseDeclarativePgDeltaPull(usePgDeltaDiff)
 			return pull.Run(cmd.Context(), schema, flags.DbConfig, name, useDeclarativePgDelta, usePgDeltaDiff, pullDiffer, afero.NewOsFs())
 		},
 		PostRun: func(cmd *cobra.Command, args []string) {
@@ -358,6 +358,13 @@ func shouldUsePgDelta() bool {
 	return utils.IsPgDeltaEnabled() || usePgDelta || viper.GetBool("EXPERIMENTAL_PG_DELTA")
 }
 
+func shouldUseDeclarativePgDeltaPull(usePgDeltaDiff bool) bool {
+	if usePgDeltaDiff {
+		return false
+	}
+	return shouldUsePgDelta()
+}
+
 func init() {
 	// Build branch command
 	dbBranchCmd.AddCommand(dbBranchCreateCmd)

apps/cli-go/cmd/db_pull_routing_test.go

@@ -0,0 +1,38 @@
+package cmd
+
+import (
+	"testing"
+
+	"github.com/spf13/viper"
+	"github.com/stretchr/testify/assert"
+)
+
+func TestShouldUseDeclarativePgDeltaPull(t *testing.T) {
+	t.Run("diff-engine pg-delta keeps the migration-file workflow", func(t *testing.T) {
+		usePgDelta = false
+		t.Cleanup(func() { usePgDelta = false })
+		assert.False(t, shouldUseDeclarativePgDeltaPull(true))
+	})
+
+	t.Run("no flag and no config means not declarative", func(t *testing.T) {
+		usePgDelta = false
+		t.Cleanup(func() { usePgDelta = false })
+		assert.False(t, shouldUseDeclarativePgDeltaPull(false))
+	})
+
+	t.Run("experimental config enables declarative", func(t *testing.T) {
+		usePgDelta = false
+		viper.Set("EXPERIMENTAL_PG_DELTA", true)
+		t.Cleanup(func() {
+			usePgDelta = false
+			viper.Set("EXPERIMENTAL_PG_DELTA", false)
+		})
+		assert.True(t, shouldUseDeclarativePgDeltaPull(false))
+	})
+
+	t.Run("use-pg-delta flag forces declarative", func(t *testing.T) {
+		usePgDelta = true
+		t.Cleanup(func() { usePgDelta = false })
+		assert.True(t, shouldUseDeclarativePgDeltaPull(false))
+	})
+}

apps/cli-go/docs/supabase/db/pull.md

@@ -8,6 +8,30 @@ Requires your local project to be linked to a remote database by running `supaba
 
 Optionally, a new row can be inserted into the migration history table to reflect the current state of the remote database.
 
-If no entries exist in the migration history table, `pg_dump` will be used to capture all contents of the remote schemas you have created. Otherwise, this command will only diff schema changes against the remote database, similar to running `db diff --linked`.
+If no entries exist in the migration history table, the default diff engine uses `pg_dump` to capture all contents of the remote schemas you have created. Otherwise, this command will only diff schema changes against the remote database, similar to running `db diff --linked`.
 
-Pass `--diff-engine pg-delta` to keep the migration-file `db pull` workflow while using pg-delta for the shadow diff step. Pass `--use-pg-delta` to switch to the declarative pg-delta export workflow instead.
+Pass `--diff-engine pg-delta` to keep the migration-file `db pull` workflow while using pg-delta for the shadow diff step. On initial pull, pg-delta replaces `pg_dump` and produces the full migration from the shadow diff alone. Pass `--use-pg-delta` to switch to the declarative pg-delta export workflow instead.
+
+When `[experimental.pgdelta] enabled = true` is set in `config.toml`, `db pull` defaults to the declarative export path. Explicit `--diff-engine pg-delta` still selects the migration-file workflow.
+
+When pulling from a remote database with `--db-url`, prefer a direct connection (`db.<project-ref>.supabase.co:5432`) over the connection pooler so pg-delta can introspect the full catalog reliably.
+
+## Debugging empty pg-delta pulls
+
+If `db pull --diff-engine pg-delta` reports `No schema changes found` but you expect schema output, set `PGDELTA_DEBUG=1` before running the command. Unlike `--debug`, this keeps SSL enabled for remote Supabase connections.
+
+```sh
+PGDELTA_DEBUG=1 supabase db pull --db-url "$DATABASE_URL" --diff-engine pg-delta
+```
+
+When pg-delta returns zero statements, the CLI writes a debug bundle under `supabase/.temp/pgdelta/debug/<timestamp>/`:
+
+- `source-catalog.json` — shadow database baseline pg-delta extracted
+- `target-catalog.json` — remote database pg-delta extracted
+- `pgdelta-stderr.txt` — pg-delta script diagnostics (statement count, schemas)
+- `connection.txt` — redacted connection metadata
+- `error.txt` — error summary
+
+Catalog files are not written during normal `db pull` runs. The `.temp/pgdelta` directory is also used by migration catalog caching (`db push`, local `db start`) when `[experimental.pgdelta] enabled = true`.
+
+For TLS tracing without disabling SSL, use `SUPABASE_SSL_DEBUG=true` alongside `PGDELTA_DEBUG=1`.

apps/cli-go/internal/db/declarative/debug.go

@@ -18,12 +18,16 @@ const (
 
 // DebugBundle collects diagnostic artifacts when a declarative operation fails.
 type DebugBundle struct {
-	ID           string   // timestamp-based unique ID (e.g. "20240414-044403")
-	SourceRef    string   // path to source catalog
-	TargetRef    string   // path to target catalog
-	MigrationSQL string   // generated migration (if available)
-	Error        error    // the error that occurred
-	Migrations   []string // list of local migration files
+	ID             string   // timestamp-based unique ID (e.g. "20240414-044403")
+	SourceRef      string   // path to source catalog
+	TargetRef      string   // path to target catalog
+	SourceCatalog  string   // inline source catalog JSON (optional)
+	TargetCatalog  string   // inline target catalog JSON (optional)
+	MigrationSQL   string   // generated migration (if available)
+	PgDeltaStderr  string   // edge-runtime stderr from pg-delta scripts
+	ConnectionInfo string   // redacted connection metadata
+	Error          error    // the error that occurred
+	Migrations     []string // list of local migration files
 }
 
 // SaveDebugBundle writes diagnostic artifacts to .temp/pgdelta/debug/<ID>/ and
@@ -38,14 +42,18 @@ func SaveDebugBundle(bundle DebugBundle, fsys afero.Fs) (string, error) {
 	}
 
 	// Copy source catalog if available
-	if len(bundle.SourceRef) > 0 {
+	if len(bundle.SourceCatalog) > 0 {
+		_ = utils.WriteFile(filepath.Join(debugDir, "source-catalog.json"), []byte(bundle.SourceCatalog), fsys)
+	} else if len(bundle.SourceRef) > 0 {
 		if data, err := afero.ReadFile(fsys, bundle.SourceRef); err == nil {
 			_ = utils.WriteFile(filepath.Join(debugDir, "source-catalog.json"), data, fsys)
 		}
 	}
 
 	// Copy target catalog if available
-	if len(bundle.TargetRef) > 0 {
+	if len(bundle.TargetCatalog) > 0 {
+		_ = utils.WriteFile(filepath.Join(debugDir, "target-catalog.json"), []byte(bundle.TargetCatalog), fsys)
+	} else if len(bundle.TargetRef) > 0 {
 		if data, err := afero.ReadFile(fsys, bundle.TargetRef); err == nil {
 			_ = utils.WriteFile(filepath.Join(debugDir, "target-catalog.json"), data, fsys)
 		}
@@ -61,6 +69,14 @@ func SaveDebugBundle(bundle DebugBundle, fsys afero.Fs) (string, error) {
 		_ = utils.WriteFile(filepath.Join(debugDir, "error.txt"), []byte(bundle.Error.Error()), fsys)
 	}
 
+	if len(bundle.PgDeltaStderr) > 0 {
+		_ = utils.WriteFile(filepath.Join(debugDir, "pgdelta-stderr.txt"), []byte(bundle.PgDeltaStderr), fsys)
+	}
+
+	if len(bundle.ConnectionInfo) > 0 {
+		_ = utils.WriteFile(filepath.Join(debugDir, "connection.txt"), []byte(bundle.ConnectionInfo), fsys)
+	}
+
 	// Copy migration files
 	if len(bundle.Migrations) > 0 {
 		migrationsDir := filepath.Join(debugDir, "migrations")

apps/cli-go/internal/db/diff/diff.go

@@ -1,7 +1,6 @@
 package diff
 
 import (
-	"bytes"
 	"context"
 	"fmt"
 	"io"
@@ -22,7 +21,6 @@ import (
 	"github.com/jackc/pgconn"
 	"github.com/jackc/pgx/v4"
 	"github.com/spf13/afero"
-	"github.com/spf13/viper"
 	"github.com/supabase/cli/internal/db/start"
 	"github.com/supabase/cli/internal/pgdelta"
 	"github.com/supabase/cli/internal/utils"
@@ -33,10 +31,11 @@ import (
 type DiffFunc func(context.Context, pgconn.Config, pgconn.Config, []string, ...func(*pgx.ConnConfig)) (string, error)
 
 func Run(ctx context.Context, schema []string, file string, config pgconn.Config, differ DiffFunc, usePgDelta bool, fsys afero.Fs, options ...func(*pgx.ConnConfig)) (err error) {
-	out, err := DiffDatabase(ctx, schema, config, os.Stderr, fsys, differ, usePgDelta, options...)
+	result, err := DiffDatabase(ctx, schema, config, os.Stderr, fsys, differ, usePgDelta, options...)
 	if err != nil {
 		return err
 	}
+	out := result.SQL
 	branch := utils.GetGitBranch(fsys)
 	fmt.Fprintln(os.Stderr, "Finished "+utils.Aqua("supabase db diff")+" on branch "+utils.Aqua(branch)+".\n")
 	if err := SaveDiff(out, file, fsys); err != nil {
@@ -161,18 +160,18 @@ func MigrateShadowDatabase(ctx context.Context, container string, fsys afero.Fs,
 	return migration.ApplyMigrations(ctx, migrations, conn, afero.NewIOFS(fsys))
 }
 
-func DiffDatabase(ctx context.Context, schema []string, config pgconn.Config, w io.Writer, fsys afero.Fs, differ DiffFunc, usePgDelta bool, options ...func(*pgx.ConnConfig)) (string, error) {
+func DiffDatabase(ctx context.Context, schema []string, config pgconn.Config, w io.Writer, fsys afero.Fs, differ DiffFunc, usePgDelta bool, options ...func(*pgx.ConnConfig)) (DatabaseDiff, error) {
 	fmt.Fprintln(w, "Creating shadow database...")
 	shadow, err := CreateShadowDatabase(ctx, utils.Config.Db.ShadowPort)
 	if err != nil {
-		return "", err
+		return DatabaseDiff{}, err
 	}
 	defer utils.DockerRemove(shadow)
 	if err := start.WaitForHealthyService(ctx, utils.Config.Db.HealthTimeout, shadow); err != nil {
-		return "", err
+		return DatabaseDiff{}, err
 	}
 	if err := MigrateShadowDatabase(ctx, shadow, fsys, options...); err != nil {
-		return "", err
+		return DatabaseDiff{}, err
 	}
 	shadowConfig := pgconn.Config{
 		Host:     utils.Config.Hostname,
@@ -189,20 +188,20 @@ func DiffDatabase(ctx context.Context, schema []string, config pgconn.Config, w
 				declDir := utils.GetDeclarativeDir()
 				if exists, _ := afero.DirExists(fsys, declDir); exists {
 					if err := pgdelta.ApplyDeclarative(ctx, config, fsys); err != nil {
-						return "", err
+						return DatabaseDiff{}, err
 					}
 				} else {
 					if err := migrateBaseDatabase(ctx, config, declared, fsys, options...); err != nil {
-						return "", err
+						return DatabaseDiff{}, err
 					}
 				}
 			} else {
 				if err := migrateBaseDatabase(ctx, config, declared, fsys, options...); err != nil {
-					return "", err
+					return DatabaseDiff{}, err
 				}
 			}
 		} else if err != nil {
-			return "", err
+			return DatabaseDiff{}, err
 		}
 	}
 	// Load all user defined schemas
@@ -211,7 +210,28 @@ func DiffDatabase(ctx context.Context, schema []string, config pgconn.Config, w
 	} else {
 		fmt.Fprintln(w, "Diffing schemas...")
 	}
-	return differ(ctx, shadowConfig, config, schema, options...)
+	if IsPgDeltaDebugEnabled() && usePgDelta {
+		// Capture the shadow baseline catalog and edge-runtime stderr so an
+		// empty diff can be inspected later. DiffPgDeltaRefDetailed mirrors the
+		// pg-delta differ but additionally surfaces stderr, which differ() drops.
+		debugCapture := &PgDeltaDebugCapture{}
+		if snapshot, exportErr := exportCatalogPgDelta(ctx, utils.ToPostgresURL(shadowConfig), "postgres", options...); exportErr == nil {
+			debugCapture.SourceCatalog = snapshot
+		} else {
+			fmt.Fprintf(w, "Warning: failed to export shadow pg-delta catalog: %v\n", exportErr)
+		}
+		result, err := DiffPgDeltaRefDetailed(ctx, utils.ToPostgresURL(shadowConfig), utils.ToPostgresURL(config), schema, pgDeltaFormatOptions(), options...)
+		if err != nil {
+			return DatabaseDiff{}, err
+		}
+		debugCapture.Stderr = result.Stderr
+		return DatabaseDiff{SQL: result.SQL, Debug: debugCapture}, nil
+	}
+	output, err := differ(ctx, shadowConfig, config, schema, options...)
+	if err != nil {
+		return DatabaseDiff{}, err
+	}
+	return DatabaseDiff{SQL: output}, nil
 }
 
 func migrateBaseDatabase(ctx context.Context, config pgconn.Config, migrations []string, fsys afero.Fs, options ...func(*pgx.ConnConfig)) error {
@@ -228,37 +248,3 @@ func migrateBaseDatabase(ctx context.Context, config pgconn.Config, migrations [
 	defer conn.Close(context.Background())
 	return migration.SeedGlobals(ctx, migrations, conn, afero.NewIOFS(fsys))
 }
-
-func diffWithStream(ctx context.Context, env []string, script string, stdout io.Writer) error {
-	cmd := utils.EdgeRuntimeStartCmd()
-	if viper.GetBool("DEBUG") {
-		cmd = append(cmd, "--verbose")
-	}
-	cmdString := strings.Join(cmd, " ")
-	entrypoint := []string{"sh", "-c", `cat <<'EOF' > index.ts && ` + cmdString + `
-` + script + `
-EOF
-`}
-	var stderr bytes.Buffer
-	if err := utils.DockerRunOnceWithConfig(
-		ctx,
-		container.Config{
-			Image:      utils.Config.EdgeRuntime.Image,
-			Env:        env,
-			Entrypoint: entrypoint,
-		},
-		container.HostConfig{
-			Binds:       []string{utils.EdgeRuntimeId + ":/root/.cache/deno:rw"},
-			NetworkMode: network.NetworkHost,
-		},
-		network.NetworkingConfig{},
-		"",
-		stdout,
-		&stderr,
-	// The "main worker has been destroyed" message may not appear at the start of stderr
-	// (e.g. preceded by other Deno runtime output), so use Contains instead of HasPrefix.
-	); err != nil && !strings.Contains(stderr.String(), "main worker has been destroyed") {
-		return errors.Errorf("error diffing schema: %w:\n%s", err, stderr.String())
-	}
-	return nil
-}