feat(query): built-in zoid/oid query operator#198
Merged
Conversation
Member
Author
|
Docs added (commit 3d7d3ba): a Diataxis how-to "Resolve relations to brains without waking objects" ( |
jensens
force-pushed
the
feat/zoid-query-operator
branch
2 times, most recently
from
July 3, 2026 07:51
aa7e6b6 to
b3e2d9d
Compare
`catalog(zoid=[...])` / `catalog(oid=[...])` filters the object_state BIGINT
primary key directly (the catalog rid IS the zoid). Lets callers resolve a set
of ZODB object ids to their security-filtered brains WITHOUT waking the objects
— e.g. zc.relation/intids: intid -> KeyReference.oid -> int.from_bytes(...) ->
zoid — replacing an object-load N+1 in relation/back-reference resolution.
- `zoid` registered as a special built-in (IndexType.ZOID) targeting the real
`zoid` column, not a JSONB idx key; `oid` is the same filter accepting raw
8-byte ZODB oids (converted via int.from_bytes, matching obj_to_zoid).
- Accepts int / digit-str / bytes; scalar or list; bool rejected; non-coercible
values dropped (ZCatalog-lenient). Explicit `::bigint[]` cast so an empty set
is `= ANY('{}')` (matches nothing) rather than a type-inference error — never
the whole table.
- No extra storage, no reindex (filters the existing PK). Composes with the
normal query and the allowedRolesAndUsers security filter injected by
searchResults. Honored by strip_non_index_keys.
Unit tests (12, no PG) + integration tests (5, real PG): list/scalar/oid-bytes,
bool rejection, empty-matches-nothing, AND-composition. Full query + columns
suites green (197 unit, 35 query-integration).
Diataxis how-to for the new zoid/oid query operator: the wake-free intid -> KeyReference.oid -> zoid -> catalog(zoid=[...]) recipe for related-items / back-reference resolution, with a full example and notes on security composition, no-reindex, empty-matches-nothing, and order preservation. Registered in the how-to toctree; adds zoid/oid/intid/IOBTree/… to the vale vocabulary.
jensens
force-pushed
the
feat/zoid-query-operator
branch
from
July 3, 2026 07:58
b3e2d9d to
6157aa8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #197.
What
Adds a built-in
zoid(andoid) query operator:Why
pgcatalog stores every record in
object_statekeyed byzoid(the BIGINT PK; the catalog rid is the zoid), but there was no way to query by it (getMetadataForRIDis unsupported). That matters whenever you hold a set of ZODB object ids and want their security-filtered brains without waking the objects — the motivating case beingzc.relation/intidsresolution:intid → getUtility(IIntIds).refs[intid].oid(wake-free)→ int.from_bytes(..., "big") → zoid → catalog(zoid=[...])On a real page (51 related objects) the wake path cost ~214 cold ZODB loads (more anonymously, walking the acquisition chain for
View); a singlecatalog(zoid=[...])returns the security-filtered brains directly. See #197 for the full analysis.How
IndexType.ZOIDpseudo-index (not a ZCatalog meta-type → never auto-registered) wired via_SPECIAL_BUILTIN_INDEX_TYPES, targeting the realzoidcolumn rather than a JSONBidxkey._handle_zoidemitszoid = ANY(%(p)s::bigint[]). Accepts int / digit-string / 8-byte oid bytes (_coerce_zoid); scalar or list;boolrejected; non-coercible values dropped (ZCatalog-lenient). Explicit::bigint[]cast so an empty set matches nothing (= ANY('{}')) instead of a type-inference error — never the whole table._MAX_ZOIDSbounds a hostile list.allowedRolesAndUserssecurity filter injected bysearchResults. Honored bystrip_non_index_keys.Tests
tests/test_query.py::TestZoidIndex— 12 unit tests (no PG): single/list ints, explicit spec, oid-bytes conversion, digit strings, bool rejection, non-coercible drop, empty-matches-nothing,_MAX_ZOIDS, AND-composition,strip_non_index_keys.tests/test_query_integration.py::TestZoidIndexIntegration— 5 integration tests (real PG): zoid list/scalar, oid-bytes, empty-matches-nothing, composition with a FieldIndex.Follow-up
Happy to add a Diataxis how-to ("resolve relations/intids to brains via zoid") if you want it in the same PR.