cli: make test Postgres connection limits configurable#2410
Draft
ZeshanA wants to merge 2 commits into
Draft
Conversation
Exposes two env vars, read when the daemon creates a test Postgres cluster, so CI runners aren't capped by the hardcoded defaults: - ENCORE_SQLDB_MAX_CONNECTIONS passes `-c max_connections=N` to the Postgres container on creation. Lifts the server-side cap above Postgres's default of 100. - ENCORE_SQLDB_POOL_SIZE replaces the hardcoded 96 total budget that infra.go divides evenly across per-package test databases. Both default to today's behaviour when unset. Raising one without the other is rarely useful: the client-side pool budget is bounded above by the server's max_connections, so CI should set them together. The server-side flag only applies when a fresh container is created (`docker start` reuses an existing container's args), which is fine for ephemeral CI runners but flagged in a comment for local dev.
Second-opinion review surfaced two issues with the previous commit: 1. The `UpdateConfig` path on ResourceManager is dead code — its only caller `Manager.generateConfig` has zero callers in the repo. All live paths (encore run, encore test for both Go and TS) flow through `RuntimeConfigGenerator.initialize()` and call `SQLDatabaseConfig()`, which returns `MaxConnections=0`. The runtime then falls back to pgx's default of 30 per pool — that's why the 100-connection server cap is hit so quickly in CI. The previous commit's env-var read in `UpdateConfig` had no runtime effect. 2. ENCORE_SQLDB_POOL_SIZE was misnamed (it's a total budget, not a per-pool size), used the global logger instead of rm.log, and didn't clamp its int result before being cast to the proto's int32 field. This commit: - Reverts `UpdateConfig` to its main-branch shape (dead but harmless). - Adds `ResourceManager.SQLDatabaseMaxConnections(numLocalDBs int) int` alongside `SQLDatabaseConfig`. Reads ENCORE_SQLDB_POOL_BUDGET, defaults to 96, logs via rm.log.Warn on invalid values, and clamps to int32 before returning. - Plumbs the helper through the live path in `runtime_config2.go`: the generator counts local (non-external) DBs once, calls the helper, and sets MaxConnections on each emitted SQLConnectionPool only when SQLDatabaseConfig returned zero, so a future non-zero override still wins. - Improves the docker.go comment to frame the daemon-lifetime caveat (the daemon's env is frozen at startup; the container reuse is one consequence of that). - Adds table-driven unit tests for SQLDatabaseMaxConnections covering defaults, custom budgets, floor-at-1, zero/negative DB counts, invalid env values, and int32 overflow clamping. Locally verified end-to-end with a three-service app: - ENCORE_SQLDB_MAX_CONNECTIONS=500 ENCORE_SQLDB_POOL_BUDGET=480 → container launched with `-c max_connections=500`; `SHOW max_connections` returns 500; each DB's pool gets 160 (= 480/3). - Both vars unset → container gets no `-c max_connections`; server reports 100; each DB's pool gets 32 (= 96/3).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two env vars, read at cluster creation time by the Encore daemon, so CI runners aren't capped by hardcoded defaults when running many parallel test packages:
ENCORE_SQLDB_MAX_CONNECTIONS— passed to the Postgres container as-c max_connections=Non creation. Lifts the server-side cap above Postgres's default of 100.ENCORE_SQLDB_POOL_BUDGET— total pgx connection budget divided evenly across the local-cluster's databases. Replaces the effective runtime default (pgx's internal 30 per pool) with a configurable total-budget-divided-by-N.Both default to current behaviour when unset — no migration needed.
Motivation
On CI runners with significant CPU/memory headroom,
encore test -p=N ./...tripsSQLSTATE 53300(too many connections) long before the hardware is saturated. The Postgres container launches with stockmax_connections=100, and the live runtime-config path (RuntimeConfigGenerator.initialize→SQLDatabaseConfig) returnsMaxConnections=0, so the Go runtime falls back to pgx's default of 30 per pool — each service eating a 30-conn slice of a 100-conn server.(Note:
ResourceManager.UpdateConfigininfra.gostill contains a96/Nheuristic, but it's dead code —Manager.generateConfighas zero callers. Out of scope for this PR; leaving it for a follow-up cleanup.)Design notes
max_connections, so CI generally sets them together (e.g.ENCORE_SQLDB_MAX_CONNECTIONS=500 ENCORE_SQLDB_POOL_BUDGET=480). Keeping them as separate vars avoids hard-coding a server-vs-client ratio.cli/cmd/encore/test.gousesDisableFlagParsingand forwards everything togo test. Threading a new flag would mean touching thedaemonpb.TestRequestproto + daemon + driver plumbing. Env vars read at the edges match the existingENCORE_SQLDB_HOST/DATABASE/USER/PASSWORDfamily incli/cmd/encore/daemon/daemon.goand are trivial to set in CI.runtime_config2.goonly whenSQLDatabaseConfig()returnedMaxConnections=0, so a future explicit per-DB override still wins.ENCORE_SQLDB_POOL_BUDGETparses to a Gointbut the proto field isint32; the helper clamps before returning to avoid silent wrap on pathological inputs.ENCORE_SQLDB_MAX_CONNECTIONSadditionally only takes effect on a fresh container —docker startreuses an existing container's args. Fine for ephemeral CI runners; flagged in code comments for local dev.Code layout
cli/daemon/run/infra/infra.go: new helperSQLDatabaseMaxConnections(numLocalDBs int) intalongsideSQLDatabaseConfig.cli/daemon/run/runtime_config2.go: adds the helper to theinfraManagerinterface; counts local (non-external) DBs once ininitialize(); applies the per-DB cap at theSQLConnectionPoolconstruction site.cli/daemon/sqldb/docker/docker.go: appends-c max_connections=Nto thedocker runargs when the env var is set.cli/daemon/run/infra/infra_test.go: table-driven tests covering defaults, custom budgets, floor-at-1, zero/negative counts, invalid env values, and int32 overflow clamping.Example usage
ENCORE_SQLDB_MAX_CONNECTIONS=500 \ ENCORE_SQLDB_POOL_BUDGET=480 \ encore test -p=8 ./...Local verification
Confirmed end-to-end with a three-service sample app:
SHOW max_connections-c fsync=off -c max_connections=500500160(= 480/3)-c fsync=off10032(= 96/3)Unit tests: 12 cases pass (
go test ./cli/daemon/run/infra/...).Test plan
go build ./cli/...passesgo test ./cli/daemon/run/... ./cli/daemon/sqldb/...passesdocker inspectandpsql SHOW max_connectionson a 3-service sample apprm.logused for warnings (carriesapp_idcontext, matching file conventions)Notes for reviewers
96/Nheuristic inResourceManager.UpdateConfigis dead code. Happy to remove it in a follow-up if desired.externaldetection (g.DefinedSecrets["sqldb::"+db.Name]) is duplicated just to count local DBs. There's no existing helper in therunlayer;cluster-sideIsExternalDBuses the samesqldb::secret-membership check. Can extract into a small helper if preferred.