@@ -10,6 +10,22 @@ cannot eliminate. Pruning is conservative: a partition is removed only when its
1010provably cannot contain a matching row, so results are always identical to running
1111without it.
1212
13+ > ### ⚠️ Many partitions? Raise ` max_locks_per_transaction ` first
14+ >
15+ > Pruning (and the index build) on a non-key column requires PostgreSQL to lock ** every**
16+ > partition in one transaction. On the ** default ` max_locks_per_transaction = 64 ` ** this
17+ > exhausts the lock table at roughly ** a few thousand partitions** , with:
18+ >
19+ > ```
20+ > ERROR: out of shared memory
21+ > HINT: You might need to increase "max_locks_per_transaction".
22+ > ```
23+ >
24+ > If you have thousands of partitions, **raise `max_locks_per_transaction` (it requires a
25+ > restart) before creating the index or querying** — see
26+ > [Scaling and partition count](#scaling-and-partition-count) for sizing. This is a
27+ > PostgreSQL limit on wide non-key access, not specific to this extension.
28+
1329## Quick Start
1430
1531Summaries are built and maintained through a custom index access method, so pruning
@@ -168,14 +184,38 @@ planner hook that prunes a non-key column before expansion, so this O(n) cost is
168184
169185Two practical consequences and how to handle them:
170186
171- - ** Lock table exhaustion (a hard wall, ~ 10k partitions on defaults).** Any query touching
172- a non-key column must lock * every* partition (and its indexes) while planning. With the
173- default ` max_locks_per_transaction = 64 ` , a query over ~ 10,000 partitions fails with
174- ` ERROR: out of shared memory ` / ` You might need to increase "max_locks_per_transaction" ` .
175- This is a PostgreSQL limit on wide non-key scans, not specific to this extension (native
176- key pruning avoids it by never locking pruned partitions). ** Mitigation:** raise
177- ` max_locks_per_transaction ` (e.g. to a few thousand) and restart — it preallocates
178- shared memory for the lock table, pushing the wall out in proportion.
187+ - ** Lock table exhaustion (the hard wall).** Two operations lock * every* partition (and its
188+ indexes) in a single transaction:
189+ - ** ` CREATE INDEX … USING table_range ` ** , which builds one child index per partition — so
190+ on too many partitions the index can't even be * built* (it fails and rolls back, leaving
191+ no summary, so queries then scan everything);
192+ - ** any query on a non-key column** , whose planning expands and locks all partitions.
193+
194+ On the default ** ` max_locks_per_transaction = 64 ` ** the lock table holds only ~ 6,400 locks
195+ (with default ` max_connections = 100 ` ), so at roughly ** a few thousand partitions** — where
196+ ` 2 × partitions ` exceeds that — you get:
197+
198+ ```
199+ ERROR: out of shared memory
200+ HINT: You might need to increase "max_locks_per_transaction".
201+ ```
202+
203+ This is a PostgreSQL limit on wide non-key access, not specific to this extension — native
204+ key pruning avoids it by never locking pruned partitions.
205+
206+ ** How to fix it.** Raise ` max_locks_per_transaction ` . It is a postmaster-level setting, so
207+ it ** requires a restart** :
208+
209+ ``` sql
210+ ALTER SYSTEM SET max_locks_per_transaction = 4096 ; -- then restart PostgreSQL
211+ ```
212+
213+ Sizing: the lock table holds about `max_locks_per_transaction × (max_connections +
214+ max_prepared_transactions)` locks, and one statement over * N* partitions needs roughly
215+ ` 2 × N ` of them (a heap + an index lock per partition). Pick a value so that product
216+ comfortably exceeds ` 2 × N ` for your largest partitioned table, with headroom for
217+ concurrency. With default ` max_connections ` , a few thousand (e.g. ` 4096 ` ) covers tens of
218+ thousands of partitions; each lock slot costs only a few hundred bytes of shared memory.
179219- ** Planning time grows with partition count.** Even below the lock wall, planning scales
180220 linearly — though the per-partition constant is now small (~ 3–4 µs warm, on par with
181221 ` CHECK ` constraint exclusion) thanks to the per-plan compilation and backend summary
@@ -244,11 +284,12 @@ range-type tests, which exercise the same code path.
244284
245285## Limitations
246286
247- - ** Planning is O(partitions)** for non-key predicates, with a hard lock-table wall around
248- ~ 10k partitions on default settings. See [ Scaling] ( #scaling-and-partition-count ) for the
249- cause and mitigations (` max_locks_per_transaction ` , prepared statements, fewer/larger
250- partitions). Native partition-key pruning does not have this limit; table_range is for
251- the cases native pruning cannot handle.
287+ - ** Lock-table wall on many partitions.** Both ` CREATE INDEX … USING table_range ` and
288+ queries on a non-key column lock every partition at once, so on the default
289+ ` max_locks_per_transaction = 64 ` they fail (` out of shared memory ` ) at roughly a few
290+ thousand partitions. ** Raise ` max_locks_per_transaction ` (needs a restart)** — see
291+ [ Scaling] ( #scaling-and-partition-count ) for sizing. Native partition-key pruning does not
292+ have this limit; table_range is for the cases native pruning cannot handle.
252293- Pruning is a ** planning-time cost / execution-time win** tradeoff: on small partitions
253294 the per-plan overhead can exceed the scan it saves. Measure with
254295 ` table_range.enable_pruning ` .
0 commit comments