Close read txn at request end (afterCompletion) + idle-in-xact safety net (#118)#58
Merged
Conversation
…etion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… end (#118) ZODB calls _storage.afterCompletion() after every transaction commit/abort and on Connection.close(). Previously PGJsonbStorageInstance did not define it, so the hook was a silent no-op and the REPEATABLE READ snapshot opened by poll_invalidations() persisted across requests. Each lingering read tx held a virtualxid that blocks CREATE INDEX CONCURRENTLY indefinitely under load. The new method delegates to _end_read_txn() (idempotent) and never propagates exceptions — a connection killed externally just gets logged; the next operation rebuilds via the pool. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nn, end-to-end) The end-to-end test verifies that after Connection.close(), the backend's pg_stat_activity state is 'idle' (not 'idle in transaction') — exactly the conditions that previously blocked CIC. Note: mid-request transaction.commit/abort fires our afterCompletion, but ZODB.Connection.afterCompletion immediately follows up with newTransaction() → poll_invalidations() which re-opens the read tx for the next implicit transaction. The fix's actual win is at Connection.close() (pool return / request end), which the end-to-end test pins down. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…out (#118) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Defense in depth for any leaked virtualxid that bypasses afterCompletion (SIGKILL-ed worker, third-party plugin, etc.). Default 60_000 ms, env-overridable via ZODB_PGJSONB_IDLE_IN_XACT_TIMEOUT_MS. Set to 0 to disable. Replaces the inline `lambda` configure with a named helper that also applies the timeout setting alongside autocommit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jensens
added a commit
to bluedynamics/plone-pgcatalog
that referenced
this pull request
Apr 13, 2026
release_request_connection() now issues an explicit conn.rollback() before pool.putconn(conn). Pool conns aren't autocommit by default, so any prior SELECT left an implicit transaction open, holding a virtualxid that blocks CREATE INDEX CONCURRENTLY. rollback() on a conn with no open txn is a cheap no-op; on autocommit conns it's also a no-op. Wrapped in contextlib.suppress so a dead conn doesn't break pool return. Companion to bluedynamics/zodb-pgjsonb#58 (storage-conn path). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stops `PGJsonbStorageInstance` leaking REPEATABLE READ transactions across requests, so they no longer block `CREATE INDEX CONCURRENTLY` and similar maintenance operations.
Closes bluedynamics/plone-pgcatalog#118.
Why mid-request transactions don't appear to benefit
ZODB's `Connection.afterCompletion` calls `storage.afterCompletion()` then immediately calls `newTransaction()` → `poll_invalidations()` for the next implicit transaction — which re-opens the read tx. So intra-request, the read tx stays effectively open. The actual win is at `Connection.close()` (pool return / request end), where there's no follow-up `newTransaction`. End-to-end test pins this down via `pg_stat_activity` showing `state = 'idle'` after close.
Tests
Full suite: 456 passed (was 449), no regressions.
Spec
`docs/superpowers/specs/2026-04-13-idle-in-xact-design.md` — full root-cause analysis, trade-off table (Fix 1 / 2 / 3 / 4 / E), test plan.
🤖 Generated with Claude Code