@@ -45,24 +45,32 @@ Implementation:
4545New method on ` CatalogSearchResults ` :
4646
4747``` python
48- def prefetch_objects (self ):
49- """ Prefetch ZODB objects for all brains in this result set .
48+ def _prefetch_objects (self , start_index , batch_size = 100 ):
49+ """ Prefetch ZODB objects for a batch of brains .
5050
5151 Warms the storage _load_cache so subsequent getObject() calls
5252 hit the cache instead of making individual SQL queries.
53+
54+ Only prefetches `batch_size` brains starting from `start_index`,
55+ not the entire result set. Safe for large result sets where
56+ only a page is rendered (e.g. b_size=20 out of 10,000).
5357 """
5458```
5559
5660Implementation:
5761
58- 1 . Collect all zoids from brains in the result set
59- 2 . Convert to oids via ` p64(zoid) `
62+ 1 . Slice brains from ` start_index ` to ` start_index + batch_size `
63+ 2 . Collect zoids from the slice, convert to oids via ` p64(zoid) `
60643 . Get the storage instance via ` catalog._p_jar._storage `
61- 4 . Call ` storage.load_multiple(oids) `
62- 5 . Mark result set as prefetched (` _objects_prefetched = True ` )
65+ 4 . Call ` storage.load_multiple(oids) ` --- fills storage cache
66+ 5 . Track prefetched range to avoid re-fetching
67+
68+ Trigger: automatically on ` brain.getObject() ` . When a brain at
69+ index N calls getObject(), prefetch brains N to N+batch_size.
70+ If index N is already in a prefetched range, skip. Same pattern
71+ as lazy ` _load_idx_batch() ` but window-based instead of all-at-once.
6372
64- Trigger: automatically on the first ` brain.getObject() ` call in
65- the result set, same pattern as lazy ` _load_idx_batch() ` .
73+ Configurable via ` PGCATALOG_PREFETCH_BATCH ` env var (default 100).
6674
6775### What does NOT change
6876
@@ -84,9 +92,10 @@ the result set, same pattern as lazy `_load_idx_batch()`.
8492
8593| Scenario | Before | After |
8694| ----------| --------| -------|
87- | Collection (50 items) | 50 roundtrips | 1 batch + 50 cache hits |
88- | /veranstaltungen (3000 loads) | 3000 roundtrips | ~ 50-100 batches + rest from cache |
89- | Second page load (warm cache) | Cache hits | Same (no change) |
95+ | Collection (20 of 10k shown) | 20 roundtrips | 1 batch(100) + 20 cache hits |
96+ | Collection (50 items total) | 50 roundtrips | 1 batch(100) + 50 cache hits |
97+ | /veranstaltungen (3000 loads) | 3000 roundtrips | 30 batches(100) + rest from ZODB cache |
98+ | Second page load (warm cache) | ZODB cache hits | Same (no change) |
9099
91100## Implementation steps
92101
0 commit comments