feat(api-headless-cms): postgre, utils and os#5431
Draft
brunozoric wants to merge 37 commits into
Draft
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ture from ddb-es Moves 77 files: 9 feature directories (field indexing, filtering, sorting, FTS, query/body/sort modifiers, value search), elasticsearch operations directory (22 files), helpers, values, configurations, elasticsearch index lifecycle, and transformEntryToIndex. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…, recordType Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…kage All OpenSearch-related imports now point to @webiny/api-headless-cms-utils-os instead of local ~/features/, ~/operations/entry/elasticsearch/, ~/helpers/, ~/configurations, ~/elasticsearch/ paths. CmsIndexEntry type import split from ~/types.js to utils-os. Added utils-os as dependency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed 76 files: 9 feature directories, operations/entry/elasticsearch/, helpers/, values/, configurations.ts, elasticsearch/, and transformEntryToIndex.ts. All now live in @webiny/api-headless-cms-utils-os. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test files referenced extracted modules via ~/features/, ~/configurations, ~/operations/entry/elasticsearch/ paths. Updated all 31 test files to import from @webiny/api-headless-cms-utils-os. All 110 tests pass. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ckage Scaffolds the package mirroring api-sync-ddb-to-opensearch: PgOperationsBuilder converts PgWalChangeRecord rows into OS bulk operations, PgToOpenSearchFeature wires the base sync features (ExecuteSync, ExecuteSyncWithRetry, SynchronizationBuilder), and createPgToOpenSearchHandler exposes a plain async function `(records) => Promise<void>` for the future WAL listener to call, with zero AWS/Lambda dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CrsJhKekhd17sQDTkCugfF
…ormat
PgWalChangeRecord.data is now {compression, value} instead of raw string.
Matches DDB adapter pattern where compression format is self-describing.
Tests use CompressionHandler.compress() directly instead of hardcoded jsonpack.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scaffolds the new PG+OpenSearch CMS storage package (zero AWS deps) and implements SyncTableManager, which lazily creates the cms_os_sync table used to stage entry changes for OS sync. Mirrors EntryTableManager's pattern exactly: memoized in-flight ensureTable(), reset() for test cleanup, and self-registration on globalThis.__sqlTableManagers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…n + OS list Wraps @webiny/api-headless-cms-sql entry storage operations: every write op (create, createRevisionFrom, update, publish, unpublish, move, moveToBin, restoreFromBin, delete, deleteRevision, deleteMultipleEntries) delegates to SQL then writes/removes the corresponding sync-table rows via SyncWriter. list/get/getUniqueFieldValues query OpenSearch directly, mirroring the ddb-es package's OS query pattern. Point reads (getByIds, getRevisions, etc.) pass straight through to the SQL operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…istration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…penSearch sync bridge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Simplify vitest config to only load pg-os presets (not ddb-es) - Fix syncWriter test index assertion to use toContain (accounts for OS prefix) - Add pg-os keyword for preset discovery Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…reate as DI feature Replace bare createElasticsearchIndex function with a proper DI abstraction that resolves OpenSearchClient and CmsEntryOpenSearchIndex[] from the container. Both ddb-es and pg-os consumers now resolve and call indexCreate.execute() instead of passing client/indexConfigs manually. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…elete as DI feature Replace bare deleteElasticsearchIndex function with a proper DI abstraction that resolves OpenSearchClient from the container. Both ddb-es and pg-os consumers now resolve and call indexDelete.execute(). Old bare functions in elasticsearch/ directory deleted — no remaining callers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ilder as DI feature
Replace createElasticsearchBody bare function with a proper DI abstraction
that resolves all 10 OpenSearch registries/modifiers from the container.
Both ddb-es and pg-os callers now resolve bodyBuilder once and call
bodyBuilder.build({ model, params }) — down from 12+ params per call site.
Removed ~8 unused registry resolutions and type imports from each consumer.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…athFactory as DI features Add CmsEntryOpenSearchExecFiltering (wraps filtering pipeline with operatorRegistry, valueSearchRegistry, filterRegistry from DI) and CmsEntryOpenSearchFieldPathFactory (wraps field path computation with valueSearchRegistry from DI). Update body.ts and sort.ts to accept these DI features as params. BodyBuilderImpl now resolves ExecFiltering and FieldPathFactory from container instead of operatorRegistry/filterRegistry directly. Both consumers register the new features. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ratorList as DI features Add CmsEntryOpenSearchValueTransformer (wraps transformValueForSearch with valueSearchRegistry from DI) and CmsEntryOpenSearchOperatorList (wraps createOperatorPluginList with operatorRegistry from DI). Update filtering pipeline (exec.ts, applyFiltering.ts) to use these DI features instead of bare functions. ExecFilteringImpl now resolves OperatorList, ValueTransformer, and FieldPathFactory from container. Updated ddb-es test helpers to register new features in test container. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Research docs from subagent analysis covering body, fields, filtering, sort, fullTextSearch, and utility files — DI candidacy assessment for each. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…plementations Move body.ts logic into BodyBuilderImpl.build(), exec.ts logic into ExecFilteringImpl.execute(). Move sort.ts, applyFiltering.ts, populated.ts, values.ts as helpers inside their respective feature folders. Delete old files: body.ts, sort.ts, exec.ts, path.ts, transformValueForSearch.ts, plugins/operator.ts, filtering/applyFiltering.ts, filtering/populated.ts, filtering/values.ts. Keep filtering/index.ts as thin compatibility shim for ddb-es test imports. Only shared utilities remain in operations/entry/ elasticsearch/ (fields, initialQuery, keyword, fullTextSearch, types, etc). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… helpers Delete filtering/index.ts from utils-os (was only used by ddb-es tests). Move CreateExecFilteringResponse type and createExecFiltering helper into ddb-es test mock. Update 8 test files to import from local mock instead of cross-package path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…UtilsFeature Create single CmsEntryOpenSearchUtilsFeature that registers all 11 utils-os features. Consumers now call one .register() instead of 11. Remove Impl exports from barrel — only abstractions and the composite feature are public API. Update ddb-es and pg-os feature files and test container to use the composite feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…bstractions Add exports/api/cms/opensearch.ts as the canonical import path for all OpenSearch abstractions. Update ddb-es and pg-os to import from @webiny/api-headless-cms-utils-os/exports/api/cms/opensearch.js instead of deep feature paths. ddb-es re-export file updated to delegate to the canonical path. Merged duplicate imports in all consumer files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…feature Barrel index.ts now exports only CmsEntryOpenSearchUtilsFeature. Abstractions are available via exports/api/cms/opensearch.js path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix api-headless-cms-es-tasks to import configurations and abstractions from @webiny/api-headless-cms-utils-os instead of ddb-es (which moved these in Phase 1). Drop unused ddb-es dep from es-tasks. Fix webiny package to import OpenSearch abstractions from ddb-es canonical exports path instead of deep feature paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ws package Remove AWS SigV4 signing from base api-opensearch client so non-AWS deployments (PG+OS with self-hosted OpenSearch) don't pull AWS SDK dependencies. New api-opensearch-aws package provides createAwsOpenSearchClient wrapper and AwsOpenSearchClientFactoryFeature for DI factory replacement. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AWS OpenSearch package is internal wiring, not user-facing API. Users interact with base OpenSearch abstraction; AWS feature registers automatically. Removed exports/ folder and regenerated webiny package. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Extract OS stuff into utils-cms package.
Extract OS packages: api-opensearch, api-opensearch-aws, api-opensearch-server.
Use base pg cms package and decorate with opensearch operations.
How Has This Been Tested?
Manually and vitest.