Skip to content
Open
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
dc8875d
feat: add config package with TOML parsing and LoadConfig
William-W-Chen May 14, 2026
8d14729
test: add config merge, boolean override, and error case tests
William-W-Chen May 14, 2026
2213d48
feat: add --config and --env flags to root command
William-W-Chen May 14, 2026
4c3958f
feat: plan/apply/dump commands read config values as fallback for CLI…
William-W-Chen May 14, 2026
b92078e
feat: add multi-tenant schema loop for plan and apply commands
William-W-Chen May 14, 2026
cf1cac8
test: add integration tests for config file loading and env overrides
William-W-Chen May 14, 2026
dd416f9
docs: update comments for ResolvedConfig, envConfig, and fileConfig s…
William-W-Chen May 14, 2026
e6fe1f3
feat: add configuration file support with environment overrides and m…
William-W-Chen May 14, 2026
a837a08
feat: implement read-only transaction for schema discovery queries
William-W-Chen May 14, 2026
5b23197
feat: add tests for read-only transaction enforcement in schema disco…
William-W-Chen May 14, 2026
9a57b5a
refactor: streamline CI workflows for unit and integration tests with…
William-W-Chen May 14, 2026
d4dca47
feat: update Docker workflow to use GitHub Container Registry
William-W-Chen May 14, 2026
03e59de
fix: ensure integration tests depend on unit tests in CI workflow
William-W-Chen May 14, 2026
f674c42
refactor: streamline CI workflows for unit and integration tests with…
William-W-Chen May 14, 2026
6df6f1a
fix: ensure integration tests depend on unit tests in CI workflow
William-W-Chen May 14, 2026
3130e92
refactor: consolidate unit and integration tests into a single CI job
William-W-Chen May 14, 2026
413e19a
revert: undo unnecessary trailing newline change in ci-test.yml
William-W-Chen May 14, 2026
3e2b293
Merge branch 'ci/matrix-improvements' into feat/config-file
William-W-Chen May 14, 2026
0b07702
refactor: remove integration test job from CI workflow
William-W-Chen May 14, 2026
282b531
Merge pull request #1 from NFUChen/feat/config-file
NFUChen May 14, 2026
9b5d90f
refactor: enhance PreRunE hooks to apply configuration for apply, dum…
William-W-Chen May 14, 2026
f4e4d43
feat: enhance PreRunE hooks to apply configuration for apply, dump, a…
William-W-Chen May 14, 2026
a0c9516
revert: restore changes from upstream base main
William-W-Chen May 14, 2026
d058178
refactor: remove applyConfigToDump call from runDump function
William-W-Chen May 14, 2026
b3c5463
fix: skip multi-schema path when --plan flag is used in apply
William-W-Chen May 15, 2026
2010913
fix: use URL-encoded DSN in DiscoverSchemas to prevent injection
William-W-Chen May 15, 2026
bd11fcc
fix: apply plan DB env vars in runPlanMultiSchema
William-W-Chen May 15, 2026
56c1b23
fix: apply plan DB env vars in runApplyMultiSchema
William-W-Chen May 15, 2026
cfc80b8
fix: redirect multi-schema progress banners to stderr
William-W-Chen May 15, 2026
953a0e5
fix: use per-schema output filenames to prevent overwrite in multi-sc…
William-W-Chen May 15, 2026
d5a78ea
fix: clear resolved config when config file is absent
William-W-Chen May 15, 2026
268e82c
test: add tests for deriveSchemaOutputTarget and --plan multi-schema …
William-W-Chen May 15, 2026
2073159
feat: implement multi-schema plan handling and output processing
William-W-Chen May 15, 2026
19c132f
feat: refactor output processing to use Outputter interface for plans
William-W-Chen May 15, 2026
49e13fa
feat: add human-readable preview output for multi-schema plans
William-W-Chen May 15, 2026
3a34820
Refactor privilege and schema management plans to standardize JSON st…
William-W-Chen May 15, 2026
7baa7b0
Refactor migration plan structure and related functions
William-W-Chen May 15, 2026
51b0176
chore: rename go file
William-W-Chen May 15, 2026
4e4d7bc
refactor: rename GeneratePlan to GenerateSchemaPlan for clarity
William-W-Chen May 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,75 @@ If the build fails with a `vendorHash` mismatch, update `nix/pgschema.nix` with
- [Docs](https://www.pgschema.com)
- [GitHub issues](https://github.com/pgplex/pgschema/issues)

## Configuration file

Instead of passing flags every time, you can create a `pgschema.toml` config file:

```toml
host = "localhost"
port = 5432
db = "myapp"
user = "postgres"
schema = "public"
file = "schema.sql"
```

Then simply run:

```bash
pgschema plan
pgschema apply
```

### Named environments

Use `[env.*]` blocks to define per-environment overrides. Values inherit from the base level:

```toml
schema = "public"
file = "schema.sql"

[env.dev]
host = "localhost"
db = "myapp_dev"
user = "postgres"

[env.prod]
host = "prod-db.internal"
db = "myapp_prod"
user = "app_user"
lock-timeout = "30s"
auto-approve = false
```

```bash
pgschema plan --env dev
pgschema apply --env prod
```

### Multi-tenant schema loop

For multi-tenant setups where each tenant has its own schema, define a `[schemas]` block with a SQL query that returns schema names. `plan` and `apply` will iterate over all discovered schemas automatically:

```toml
host = "localhost"
db = "myapp"
user = "postgres"
file = "tenant.sql"

[schemas]
query = "SELECT schema_name FROM information_schema.schemata WHERE schema_name LIKE 'tenant_%'"
```

```bash
pgschema plan # plans migration for each tenant schema
pgschema apply # applies migration to each tenant schema
```

### Priority

CLI flags always take precedence: **CLI flags > env vars > config env > config base > defaults**.

## Quick example

### Step 1: Dump schema
Expand Down
179 changes: 178 additions & 1 deletion cmd/apply/apply.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"os"
"strings"

"github.com/pgplex/pgschema/cmd/config"
planCmd "github.com/pgplex/pgschema/cmd/plan"
"github.com/pgplex/pgschema/cmd/util"
"github.com/pgplex/pgschema/internal/fingerprint"
Expand Down Expand Up @@ -49,7 +50,10 @@ var ApplyCmd = &cobra.Command{
Long: "Apply a migration plan to update a database schema. Either provide a desired state file (--file) to generate and apply a plan, or provide a pre-generated plan file (--plan) to execute directly.",
RunE: RunApply,
SilenceUsage: true,
PreRunE: util.PreRunEWithEnvVarsAndConnectionAndApp(&applyDB, &applyUser, &applyHost, &applyPort, &applyApplicationName),
PreRunE: func(cmd *cobra.Command, args []string) error {
applyConfigToApply(cmd)
return util.PreRunEWithEnvVarsAndConnectionAndApp(&applyDB, &applyUser, &applyHost, &applyPort, &applyApplicationName)(cmd, args)
},
}

func init() {
Expand Down Expand Up @@ -266,6 +270,12 @@ func ApplyMigration(config *ApplyConfig, provider postgres.DesiredStateProvider)

// RunApply executes the apply command logic. Exported for testing.
func RunApply(cmd *cobra.Command, args []string) error {
cfg := config.Get()
if cfg != nil && cfg.Schemas != nil && cfg.Schemas.Query != "" &&
!cmd.Flags().Changed("schema") && !cmd.Flags().Changed("plan") {
return runApplyMultiSchema(cmd, cfg)
Comment thread
NFUChen marked this conversation as resolved.
Comment thread
NFUChen marked this conversation as resolved.
}

// Validate that either --file or --plan is provided
if applyFile == "" && applyPlan == "" {
return fmt.Errorf("either --file or --plan must be specified")
Expand Down Expand Up @@ -489,6 +499,173 @@ func executeGroupIndividually(ctx context.Context, conn *sql.DB, group plan.Exec
return nil
}

func runApplyMultiSchema(cmd *cobra.Command, cfg *config.ResolvedConfig) error {
// Apply plan DB environment variables (same as single-schema path)
util.ApplyPlanDBEnvVars(cmd, &applyPlanDBHost, &applyPlanDBDatabase, &applyPlanDBUser, &applyPlanDBPassword, &applyPlanDBPort, &applyPlanDBSSLMode)

// Validate plan database flags if plan-host is provided
if err := util.ValidatePlanDBFlags(applyPlanDBHost, applyPlanDBDatabase, applyPlanDBUser); err != nil {
return err
}

finalPassword := applyPassword
if finalPassword == "" {
if envPassword := os.Getenv("PGPASSWORD"); envPassword != "" {
finalPassword = envPassword
}
}
finalSSLMode := applySSLMode
if cmd == nil || !cmd.Flags().Changed("sslmode") {
if envSSLMode := os.Getenv("PGSSLMODE"); envSSLMode != "" {
finalSSLMode = envSSLMode
}
}

// Derive final plan database password
finalPlanPassword := applyPlanDBPassword
if finalPlanPassword == "" {
if envPassword := os.Getenv("PGSCHEMA_PLAN_PASSWORD"); envPassword != "" {
finalPlanPassword = envPassword
}
}

schemas, err := config.DiscoverSchemas(applyHost, applyPort, applyDB, applyUser, finalPassword, finalSSLMode, cfg.Schemas.Query)
if err != nil {
return err
}

if len(schemas) == 0 {
fmt.Fprintln(os.Stderr, "Warning: schema discovery query returned no schemas.")
return nil
}

if applyFile == "" {
return fmt.Errorf("--file is required for multi-schema apply")
}

var hasErrors bool
for _, schemaName := range schemas {
fmt.Fprintf(os.Stderr, "\n── Schema: %s ──────────────────────\n", schemaName)

perSchemaConfig := &ApplyConfig{
Host: applyHost,
Port: applyPort,
DB: applyDB,
User: applyUser,
Password: finalPassword,
Schema: schemaName,
File: applyFile,
AutoApprove: applyAutoApprove,
NoColor: applyNoColor,
LockTimeout: applyLockTimeout,
ApplicationName: applyApplicationName,
SSLMode: finalSSLMode,
PlanDBHost: applyPlanDBHost,
PlanDBSSLMode: applyPlanDBSSLMode,
}

planConfig := &planCmd.PlanConfig{
Host: applyHost,
Port: applyPort,
DB: applyDB,
User: applyUser,
Password: finalPassword,
Schema: schemaName,
File: applyFile,
ApplicationName: applyApplicationName,
SSLMode: finalSSLMode,
PlanDBHost: applyPlanDBHost,
PlanDBPort: applyPlanDBPort,
PlanDBDatabase: applyPlanDBDatabase,
PlanDBUser: applyPlanDBUser,
PlanDBPassword: finalPlanPassword,
PlanDBSSLMode: applyPlanDBSSLMode,
}

provider, err := planCmd.CreateDesiredStateProvider(planConfig)
if err != nil {
fmt.Fprintf(os.Stderr, "Error for schema %s: %v\n", schemaName, err)
hasErrors = true
continue
}

err = ApplyMigration(perSchemaConfig, provider)
provider.Stop()
if err != nil {
fmt.Fprintf(os.Stderr, "Error for schema %s: %v\n", schemaName, err)
hasErrors = true
}
}

fmt.Fprintf(os.Stderr, "\nSummary: %d schemas processed\n", len(schemas))
if hasErrors {
return fmt.Errorf("one or more schemas had errors")
}
return nil
}

func applyConfigToApply(cmd *cobra.Command) {
cfg := config.Get()
if cfg == nil {
return
}

if !cmd.Flags().Changed("host") && cfg.Host != "" {
applyHost = cfg.Host
}
if !cmd.Flags().Changed("port") && cfg.Port != 0 {
applyPort = cfg.Port
}
if !cmd.Flags().Changed("db") && cfg.DB != "" {
applyDB = cfg.DB
}
if !cmd.Flags().Changed("user") && cfg.User != "" {
applyUser = cfg.User
}
if !cmd.Flags().Changed("password") && cfg.Password != "" {
applyPassword = cfg.Password
}
if !cmd.Flags().Changed("schema") && cfg.Schema != "" {
applySchema = cfg.Schema
}
if !cmd.Flags().Changed("file") && cfg.File != "" {
applyFile = cfg.File
}
if !cmd.Flags().Changed("sslmode") && cfg.SSLMode != "" {
applySSLMode = cfg.SSLMode
}
if !cmd.Flags().Changed("lock-timeout") && cfg.LockTimeout != "" {
applyLockTimeout = cfg.LockTimeout
}
if !cmd.Flags().Changed("auto-approve") && cfg.AutoApprove {
applyAutoApprove = cfg.AutoApprove
}
if !cmd.Flags().Changed("application-name") && cfg.ApplicationName != "" {
applyApplicationName = cfg.ApplicationName
}
if !cmd.Flags().Changed("no-color") && cfg.NoColor {
applyNoColor = cfg.NoColor
}
if !cmd.Flags().Changed("plan-host") && cfg.PlanHost != "" {
applyPlanDBHost = cfg.PlanHost
}
if !cmd.Flags().Changed("plan-port") && cfg.PlanPort != 0 {
applyPlanDBPort = cfg.PlanPort
}
if !cmd.Flags().Changed("plan-db") && cfg.PlanDB != "" {
applyPlanDBDatabase = cfg.PlanDB
}
if !cmd.Flags().Changed("plan-user") && cfg.PlanUser != "" {
applyPlanDBUser = cfg.PlanUser
}
if !cmd.Flags().Changed("plan-password") && cfg.PlanPassword != "" {
applyPlanDBPassword = cfg.PlanPassword
}
if !cmd.Flags().Changed("plan-sslmode") && cfg.PlanSSLMode != "" {
applyPlanDBSSLMode = cfg.PlanSSLMode
}
}

// truncateSQL truncates a SQL statement for display purposes
func truncateSQL(sql string, maxLen int) string {
// Remove extra whitespace and newlines
Expand Down
45 changes: 45 additions & 0 deletions cmd/apply/apply_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"strings"
"testing"

"github.com/pgplex/pgschema/cmd/config"
"github.com/pgplex/pgschema/internal/version"
"github.com/spf13/cobra"
)
Expand Down Expand Up @@ -541,3 +542,47 @@ func TestApplyCommand_PlanDatabaseFlags(t *testing.T) {
t.Errorf("Expected default plan-password to be empty, got '%s'", planPasswordFlag.DefValue)
}
}

func TestRunApply_PlanFlagSkipsMultiSchema(t *testing.T) {
// When --plan is provided alongside a [schemas] config, RunApply must NOT enter
// the multi-schema path. If it did, it would return "--file is required for
// multi-schema apply" instead of the expected plan-loading error.

// Set up a config that has a schemas query
cfg := &config.ResolvedConfig{
Schemas: &config.SchemasConfig{Query: "SELECT schema_name FROM information_schema.schemata"},
}
config.SetResolved(cfg)
defer config.SetResolved(nil)

// Create a minimal (but invalid version) plan file to trigger a version error,
// not a "file required" error.
tmpDir := t.TempDir()
planPath := filepath.Join(tmpDir, "plan.json")
planJSON := `{"version":"0.0.0","pgschema_version":"test","created_at":"2024-01-01T00:00:00Z","transaction":true,"summary":{"total":0,"add":0,"change":0,"destroy":0,"by_type":{}},"diffs":[]}`
if err := os.WriteFile(planPath, []byte(planJSON), 0644); err != nil {
t.Fatalf("Failed to write plan file: %v", err)
}

applyDB = "testdb"
applyUser = "testuser"
applyFile = ""
applyPlan = planPath
defer func() {
applyDB = ""
applyUser = ""
applyFile = ""
applyPlan = ""
}()

// Mark --plan as explicitly set so the guard can detect it
ApplyCmd.Flags().Set("plan", planPath)
defer ApplyCmd.Flags().Set("plan", "")

err := RunApply(ApplyCmd, []string{})

// Must NOT be the multi-schema error
if err != nil && strings.Contains(err.Error(), "--file is required for multi-schema apply") {
t.Errorf("--plan flag should prevent entering multi-schema path, got: %v", err)
}
}
Loading