docs: update reference, how-to, and explanation docs for b21-b28 features (#55) (#56)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: jensens

docs/sources/explanation/performance.md

@@ -235,3 +235,31 @@ the current floor are:
   queries per request.
 - **Increase batch sizes.** For bulk operations (reindex, migration), larger batches
   amortize per-statement overhead.
+
+### Composite indexes for multi-field queries
+
+PostgreSQL rarely combines individual single-column indexes via BitmapAnd.
+When a catalog query filters on multiple fields simultaneously (the common
+case—folder listings filter on `path_parent` + `portal_type` +
+`allowedRolesAndUsers`), PG picks one index and sequentially filters the
+rest. On a 137K-object database this means 3+ second query times.
+
+plone.pgcatalog ships composite indexes for the most common patterns:
+
+- `(path_parent, portal_type)` for folder listings and navigation
+- `(path pattern, portal_type)` for collections and search
+- `(path pattern, path_depth, portal_type)` for navigation tree
+- `(portal_type, review_state)` for workflow-filtered listings
+
+Custom catalog indexes registered via GenericSetup also get btree
+expression indexes automatically at startup.
+
+### Slow query detection
+
+Queries exceeding `PGCATALOG_SLOW_QUERY_MS` (default: 10 ms) are logged
+as warnings and recorded in the `pgcatalog_slow_queries` PostgreSQL table.
+The ZMI "Slow Queries" tab aggregates these by query field pattern and
+suggests composite index DDL for frequent patterns.
+
+This is a self-tuning feedback loop: deploy the site, let it accumulate
+slow query data under real load, then add the suggested indexes.

docs/sources/explanation/tika-extraction.md

@@ -246,7 +246,11 @@ Enabling Tika does not change how existing search works:
 - **Title and Description** are still indexed synchronously during
   `catalog_object()`, with immediate availability.
 - **Rich-text body** (SearchableText from `portal_transforms`) is
-  still indexed synchronously.
+  still indexed synchronously for non-File content types. For `IFile`
+  objects, `portal_transforms` is skipped when Tika is active—the
+  expensive `pdftotext`/`wv` calls and BFS graph traversal of the
+  transform registry are avoided entirely. See
+  {doc}`../how-to/custom-blob-searchabletext` for custom types.
 - **Tika extraction** adds to the existing tsvector asynchronously.
   A brief window (seconds to minutes, depending on queue
   depth and Tika processing time) exists where the blob content is not yet

docs/sources/how-to/rebuild-catalog.md

@@ -32,7 +32,7 @@ Expected timing: approximately 15 ms per object.
 
 ## Selective reindex (reindexIndex)
 
-Re-extracts a single index for all objects:
+Re-extracts a single index from all ZODB objects:
 
 ```python
 catalog.reindexIndex("review_state")
@@ -53,7 +53,7 @@ This happens automatically and does not trigger ZODB serialization of the object
 | `clearFindAndRebuild()` | Yes | Yes | ~15 ms/obj | Schema changes, corrupt data, major upgrades |
 | `refreshCatalog(clear=0)` | No | Re-catalogs existing | ~15 ms/obj | Reindex all without losing uncataloged objects |
 | `refreshCatalog(clear=1)` | Yes | Yes | Same | Equivalent to `clearFindAndRebuild()` |
-| `reindexIndex("name")` | No (single key) | No (PG only) | Fast | Single index changed, new indexer deployed |
+| `reindexIndex("name")` | No (single key) | Yes (ZODB load) | ~5 ms/obj | Single index changed, new indexer deployed |
 
 **`clearFindAndRebuild()`** NULLs all catalog columns (path, idx,
 searchable_text, backend extras), then traverses the entire portal tree
@@ -66,11 +66,11 @@ resolves each from ZODB, and re-extracts index values.
 It does not
 discover objects that were never cataloged.
 
-**`reindexIndex("name")`** is a PostgreSQL-only operation: it reads the
-existing `idx` JSONB for all objects that have the named key and
-re-applies it.
-It does not re-extract values from the live Zope object.
-To re-extract from objects, use `refreshCatalog()`.
+**`reindexIndex("name")`** loads each cataloged object from ZODB via
+`unrestrictedTraverse`, extracts the requested index value, and writes
+a JSONB merge update. This is faster than `refreshCatalog()` because
+it only re-extracts the single requested index, not all of them.
+Available via ZMI: Indexes & Metadata tab > [reindex] button per index.
 
 ## Troubleshooting
 

docs/sources/reference/configuration.md

@@ -58,6 +58,7 @@ need a separate `%import` directive.
 | `PGCATALOG_TIKA_URL` | (none) | Tika server URL, for example `http://localhost:9998`. Enables async text extraction from binary content (PDFs, Office docs, images). When set, the queue table and merge function are created at startup. See {doc}`../how-to/enable-tika-extraction`. |
 | `PGCATALOG_TIKA_CONTENT_TYPES` | common office/PDF/image types | Comma-separated MIME types to send to Tika. Default includes PDF, MS Office, OpenDocument, RTF, and common image formats. |
 | `PGCATALOG_TIKA_INPROCESS` | (none) | Set to `true`, `1`, or `yes` to start the extraction worker as a daemon thread inside the Zope process. Requires `PGCATALOG_TIKA_URL`. |
+| `PGCATALOG_SLOW_QUERY_MS` | `10` | Threshold in milliseconds for slow query detection. Queries exceeding this are logged as warnings and recorded in the `pgcatalog_slow_queries` table for analysis via the ZMI Slow Queries tab. Set to `0` to disable. |
 | `ZODB_TEST_DSN` | `dbname=zodb_test host=localhost port=5433 user=zodb password=zodb` | DSN for test database (tests only). |
 | `BM25_TEST_DSN` | `dbname=zodb_test host=localhost port=5434 user=zodb password=zodb` | DSN for BM25 integration tests (tests only). |