Skip to content

Vocabulary consolidation (#2901): what shipped, and why Track C items 1–3 are not planned #2945

Description

@os-zhuang

✅ Resolved — every track dispositioned

Track Outcome
A — delete three orphan vocabularies objectstack-ai/objectstack#4061
B — renderer-side convergence #2974, #2940, plus the third table in #2989 and #3022
Additions-only objectstack-ai/objectstack#4070
C item 4 — AggregationFunction lockstep objectstack-ai/objectstack#4153
C items 1–3 — narrow VALID_AST_OPERATORS, retire the aliases, rename the $-tokens Not planned — see the Track C table
Prerequisites 1, 2, 4 ✅ (1 had already landed; 2 = objectstack-ai/objectstack#4107; 4 is structural + asserted)

Why items 1–3 are not planned. Each was a precondition for fixing a silent
failure
, and every one of those failures has since been fixed by a cheaper
means: the drivers throw instead of dropping (objectstack-ai/objectstack#3948),
all three of objectui's translation tables are pinned to the spec with parity
guards (#2974, #2989, #3022), saveMeta stops minting alias rows
(objectstack-ai/objectstack#4107), and the orphans are gone
(objectstack-ai/objectstack#4061). What remains is reducing the count of
accepted spellings — paid for by rewriting live customer rows, against
sys_saved_report.query_json that cannot be enumerated statically. The
risk/benefit has inverted since this was filed.

Follow-ups carrying the loose ends: objectstack-ai/objectstack#4121 (protocol
rejects a malformed filter array), objectstack-ai/objectstack#4157
(number/string/boolean metric SQL), #2997 → closed.

Rewritten after a risk assessment. The original version of this issue said the work was cheap "while spec 16 is still RC". That framing was wrong on the facts and pointed at the most dangerous change first. What follows replaces it. The three tracks below have genuinely different risk, and only one of them is safe to start today.

Corrections to the original issue

  1. The version premise was wrong. objectui is not pinned to spec 16 awaiting a major bump. All 40 packages are at 17.0.0 and every importer resolves to @objectstack/spec@17.0.0-rc.0 (chore(deps): upgrade @objectstack/* to 17.0.0-rc.0 #2950). There is no version window to race.
  2. The 16→17 gap is not an enum gap. Re-extracted both trees: of the 30 named ui/ enum exports, zero changed values. DensityModeSchema and WcagContrastLevelSchema were removed (enforce-or-remove applied — good), and ChartAggregateFunctionSchema was added. Nothing else moved.
  3. Narrowing a vocabulary is not "cheap while RC". It is loud for new authoring and completely silent for everything already stored. defineStack throws on an out-of-enum value; saveMeta returns 422 — but the metadata read path neither drops nor errors, and objectui's render path does not zod-validate view metadata at all. A narrowing lands green in CI, green in os build, and degrades quietly against customer rows.

Track A — safe, ready now: delete three orphan vocabularies

Zero consumers, verified by grep across framework packages/ + apps/:

Vocabulary Location Evidence
AggregationFunctionEnum shared/enums.zod.ts:11 Only references are its own definition and its own test. Its doc comment claims it is "used across query, data-engine, analytics, field"that is false.
FilterOperator api/websocket.zod.ts:60 Reaches EventFilterSchema, which no server file imports. The realtime path never evaluates an event filter.
ODataFilterOperatorSchema api/odata.zod.ts:217 Zero importers. OData $filter is handled by a string→JSON parse, not by this enum.

Deleting these removes three of the "nine disagreeing filter vocabularies" without touching a single runtime path. Framework-side PR, no migration, no cross-repo sequencing.

Track B — done: renderer-side convergence

Closed by #2974 and #2940. Recorded here because the original issue scoped it as spec work, and it wasn't.

The interesting part is why it was reachable. before/after are canonical VIEW_FILTER_OPERATORS members (ui/view.zod.ts:90) that VALID_AST_OPERATORS does not accept — and 8 of the 19 canonical view operators are in that position, including equals. Six were masked only because objectui's adapter happened to translate them; the two it missed returned unfiltered result sets, silently. Details and the server-side fix: objectstack-ai/objectstack#3948.

Both objectui translation tables are now pinned to the spec vocabularies in both directions, so a new spec operator fails a test rather than returning every row.

Track C — dispositioned: item 4 done, items 1–3 not planned

No longer blocked, and mostly no longer wanted. The prerequisites landed
(see below); item 4 shipped; items 1–3 are closed as not planned, because
every harm they existed to prevent has since been fixed by a cheaper means.

Original assessment kept below for the record, with each row's disposition.

Target Original risk Disposition
VALID_AST_OPERATORS Do not narrow — removing a spelling converts a working bare-triple filter into a full-table scan with no error anywhere. Not planned. The silent drop is fixed: both drivers throw (objectstack#3948 item 1), and VALID_AST_OPERATORS is now derived from AST_OPERATOR_MAP's keys, so "in one list but not the other" is structurally impossible. Narrowing would only reduce the count of accepted spellings — no remaining harm to fix, and a stored-metadata migration to pay for it.
VIEW_FILTER_OPERATORS aliases Needs migration — all ~30 are actively being written to sys_metadata, because saveMeta persists the authored body verbatim and the schema's z.preprocess result is discarded. Not planned. The write path is fixed (objectstack#4107): new saves persist canonical spellings, so the table stops growing. Every consumer already sees canonical after parse. Retiring the ~30 entries now buys a smaller spec file in exchange for rewriting live rows — and the aliases actively protect stored metadata for the day page/view props get strict validation.
The $-token set Needs a compat shim — four hand-written driver switches hardcode it, and sys_sharing_rule.criteria_json + rollup field.filter store it. Not planned. Renaming is still a migration, and drift is now guarded by filter-view-operator-parity.test.ts. Adding a token remains free; renaming buys nothing a test does not already give.
AggregationFunction (data/query.zod.ts) Three places in lockstepdataset-compiler.ts:24 a second hardcoded gate, native-sql-strategy.ts:362 a third switch. Done — objectstack#4153. This one was not cosmetic: the strategy's default: COUNT(*) meant a ninth spec aggregate would pass the compiler's gate, be advertised as supported by its error prose, then return a row count in place of the requested number, silently. Now derived from the spec with a guard that fails on a new member. A number/string/boolean metric hitting the same fallback is filed as objectstack#4157.

Precedent: d419826a8 (#3373) did this consolidation once for ViewFilterRule, and its changelog records that it also had to physically rewrite stored data. That was one plugin's seed file; the general case is customer rows.

Prerequisites — all resolved

  1. objectstack#3948 item 1 — driver throws instead of skipping. Already landed when this issue was written; verified on origin/main (sql-driver.ts throws, driver-memory has 7 throw sites, both with regression tests) and objectstack#3948 is closed. Item 2 of it — the protocol passing an unparsed array through as an opaque where — continues as objectstack#4121, and is error locality, not correctness.
  2. saveMeta persists the normalized filter subtree — objectstack#4107. graftNormalizedOperators walks the authored body and parsed.data in lockstep by structure and copies across only a changed operator, so the verbatim body (and its Studio-only auxiliary fields) survives.
  3. Scan production before writing any migration. Moot: it gated items 1–3, which are now not planned. Still the first step if a future release chooses to retire the aliases anyway — as a self-directed cleanup, not to unblock anything.
  4. Derive, don't duplicate. Partly structural rather than fixable: ui/ imports data/, so data/filter.zod.ts cannot import ui/view.zod.ts's vocabulary to derive from it. VALID_AST_OPERATORS is now derived from AST_OPERATOR_MAP, and filter-view-operator-parity.test.ts asserts the rest. The wording above is stale in one respect: that comment documents an in-repo asserted parity, not a cross-repo hand-sync.

Additions-only work that needs no migration

Safe in any order, because widening rejects nothing:

  • Promote navigation into ActionType — fully implemented at core/src/actions/ActionRunner.ts:539, named in no spec enum.
  • Promote or delete combo (plugin-charts/normalizeChartSchema.ts:63).
  • Reconcile ActionType (6) vs WidgetActionTypeSchema (5): they differ only by form, dashboard header actions funnel into the same ActionRunner that implements form, and the narrower enum rejects at validation what the shared dispatcher executes.
  • Derive ListChartConfigSchema.chartType (5) from ChartTypeSchema (19) via .extract([...]) instead of retyping it.

Do not remove members from ChartTypeSchema, ActionType, or PageComponentType. Although no framework file imports them by name, kernel/metadata-type-schemas.ts:68 maps view/dashboard/action/report/dataset to their schemas, so all of them are enforced at metadata save — a removal rejects already-stored metadata.

Refs #2901, #2231, objectstack-ai/objectstack#3948

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions