refactor(spec,plugin-security)!: QueryAST stops declaring what no executor runs (#4286) (#4294)

Tombstone `query.joins` and `query.windowFunctions` (retiredKey — authoring
either is a tsc error and the parse carries the prescription), and delete the
orphaned Join and WindowFunction schema clusters with their keys: `expand` is
the live spelling for related records, and the one live window-function door
(`SqlDriver.findWithWindowFunctions()`) consumes its own flat driver shape
that the spec vocabulary never matched. Both removals register as protocol-18
semantic migrations (`query-joins-retired`, `query-window-functions-retired`)
— QueryAST is a request shape with no stored source to rewrite (#4196
precedent). Both names stay reserved at the REST boundary while the
tombstones live (retiredKey keeps them in `keyof QueryAST`).

Mark the six search sub-flags and `AggregationNode.filter`
`[EXPERIMENTAL — not enforced]` — declared engine affordances the ADR-0061
expansion and the aggregation paths never receive.

Close the gate that let the class stay invisible: `QuerySchema` joins the
liveness ledger through SPEC_ONLY_SCHEMAS as governed type `query`
(liveness/query.json — 15 live with evidence, 7 experimental via describe
markers, 5 dead). `cursor` / `distinct` / `having` are recorded dead with
evidence pending their #4286 step-3/4 dispositions, not unilaterally removed
— `distinct` is mis-wired (its only effect is REST count-suppression) and
`having` is the strongest enforce candidate (engine.aggregate() currently
rebuilds the driver AST without it).

The FLS predicate guard drops its windowFunctions walk (the clause no longer
exists to leak through); the `having` and `aggregations[].filter` walks stay
while those members stay declared.


Claude-Session: https://claude.ai/code/session_012K2BX3WQwPBA2ZBehqJUBX

Co-authored-by: Claude <noreply@anthropic.com>

Author: os-zhuang

.changeset/query-ast-inert-request-surface.md

@@ -0,0 +1,80 @@
+---
+"@objectstack/spec": major
+"@objectstack/plugin-security": patch
+---
+
+refactor(data)!: the QueryAST request surface stops declaring what no executor runs — `joins` and `windowFunctions` removed, six search flags and `aggregations[].filter` marked experimental, and the liveness ledger now governs the query surface (#4286)
+
+#4196 removed one declared-but-inert member from `FieldNode`. Applying the same
+method to the rest of the request surface (#4286) found 12 more members of
+`QueryAST` that no executor runs — `packages/objectql`'s `engine.ts` contains
+zero reads of any of them on the query path. This change dispositions the
+mechanical tiers and closes the gate that let the class stay invisible.
+
+**Removed (tombstoned): `query.joins` and `query.windowFunctions`.**
+
+- `joins` — no engine or driver ever read it; a query carrying it silently ran
+  as a single-table query. Related-record retrieval already has a live
+  spelling: `expand`. The orphaned `JoinNode` / `JoinNodeInput` /
+  `JoinNodeSchema` / `JoinType` / `JoinStrategy` exports are deleted with the
+  key (`data/JoinNode`, `data/JoinType`, `data/JoinStrategy` leave the
+  published JSON schemas).
+- `windowFunctions` — `find()` never applied it, so every OVER clause it
+  declared was silently dropped. The one live door is the SQL driver's own
+  `findWithWindowFunctions(object, query)` (driver-level, not on the
+  `IDataDriver` contract), and its input is a flat driver shape the spec
+  vocabulary never matched — `WindowFunctionNodeSchema` declared `field` /
+  `over` / `frame` members that door never read. The `WindowFunction` /
+  `WindowSpec` / `WindowFunctionNode` exports are deleted with the key.
+
+**FROM → TO**
+
+| Was | Now |
+| :--- | :--- |
+| `joins: [{ type: 'inner', object: 'customer', on: … }]` | `expand: { customer_id: { object: 'customer', fields: ['name'] } }` |
+| `joins` for one related column | `fields: ['customer_id.name']` (dotted path) |
+| `windowFunctions: [{ function: 'rank', … }]` in a query | `aggregations` + `groupBy`, or rankings in report/dashboard metadata |
+| OVER-clause SQL from an embedder | `sqlDriver.findWithWindowFunctions(object, { windowFunctions: [{ function, alias, partitionBy?, orderBy? }] })` |
+
+The one-line fix: **delete the key**. Both are `retiredKey()` tombstones on the
+non-strict `BaseQuerySchema`, so authoring either fails `tsc` (input type
+`never`) and a query still carrying one — even as an empty array — fails to
+parse with the prescription itself. `QueryAST` is a request shape, never stored
+in stack metadata, so there is no `os migrate meta` step: the removals are
+registered as protocol-18 **semantic** migrations (`query-joins-retired`,
+`query-window-functions-retired`), the #4196 precedent.
+
+Compat note for the REST boundary: both names remain **reserved** list-query
+parameters while the tombstones live (`retiredKey()` keeps a key in
+`keyof QueryAST`, which feeds `RESERVED_LIST_QUERY_PARAMS`), so nothing changes
+for objects with fields named `joins`/`windowFunctions` — the un-reservation
+happens when the tombstones age out, and is called out in
+`metadata-protocol`'s `QUERY_AST_KEYS` comment for whoever does it.
+
+**Marked `[EXPERIMENTAL — not enforced]` (no wire or compat impact):**
+`search.fuzzy` / `operator` / `boost` / `minScore` / `language` / `highlight`
+(the ADR-0061 expansion reads only `query` + `fields`) and
+`AggregationNode.filter` (a SQL `FILTER (WHERE …)` affordance neither the SQL
+builders nor the in-memory fallback applies). Authoring one is now a
+declaration, not a silent no-op.
+
+**Deliberately NOT dispositioned here** (they want a maintainer call, #4286
+steps 3–4): `having` (the strongest enforce candidate — `engine.aggregate()`
+currently rebuilds the driver AST without it), and `cursor` / `distinct`
+(shipped SDK producers `QueryBuilder.cursor()` / `.distinct()`; `distinct` is
+mis-wired — its only observable effect is suppressing the REST list count).
+All three are recorded `dead` with evidence in the new ledger.
+
+**The gate:** `QuerySchema` joins the liveness ledger through the gate's
+`SPEC_ONLY_SCHEMAS` override (the `webhook` precedent) as governed type
+`query` — the first governance of what *callers* write into a query rather
+than what authors write into metadata files. `packages/spec/liveness/query.json`
+classifies all 27 walked members (15 live with evidence, 7 experimental via
+describe markers, 5 dead), so the next declared-but-inert request member fails
+CI instead of needing a person to notice it.
+
+`@objectstack/plugin-security` (patch): the FLS predicate guard's
+`windowFunctions` walk is pruned — the clause no longer exists to leak through.
+The `having` and `aggregations[].filter` walks stay, deliberately: those
+members remain declared, and the guard being ready is what makes enforcing
+them later safe.

content/docs/data-modeling/queries.mdx

@@ -1,6 +1,6 @@
 ---
 title: Query Syntax Cheat Sheet
-description: One-page reference for ObjectStack QuerySchema — filters, sorts, pagination, aggregations, and joins
+description: One-page reference for ObjectStack QuerySchema — filters, sorts, pagination, aggregations, and expand
 ---
 
 # Query Syntax Cheat Sheet
@@ -387,47 +387,27 @@ on it either. Post-filter grouped results on the client until this is wired up.
 
 ---
 
-## Joins
+## Joins — removed
 
 <Callout type="warn">
-`joins` is defined in `QuerySchema` (`JoinNodeSchema`) and the SQL driver advertises
-`supports.joins: true`, but no driver's `find()` actually executes a join today — the SQL,
-in-memory, and MongoDB drivers all ignore the `joins` array (there is no `.join()`/`.leftJoin()`
-call in the SQL driver's query builder). Use `expand` for relationship traversal instead.
+`query.joins` was **removed in `@objectstack/spec` 18** (#4286, ADR-0049
+enforce-or-remove): no driver's `find()` ever executed a join — the SQL, in-memory, and
+MongoDB drivers all ignored the array, so it only ever declared a capability that did
+not run. The key is tombstoned: authoring it is a `tsc` error, and a query carrying it
+(even `joins: []`) fails to parse with the upgrade prescription. The
+`JoinNode`/`JoinType`/`JoinStrategy` exports left with it.
 </Callout>
 
-### Join Types
-
-| Type | Description |
-|:---|:---|
-| `inner` | Records with matches in both objects |
-| `left` | All records from left + matching from right |
-| `right` | All records from right + matching from left |
-| `full` | All records from both objects |
-
-### Join Strategies
-
-| Strategy | Description |
-|:---|:---|
-| `auto` | Let the engine choose the best strategy |
-| `database` | Push join to the database level |
-| `hash` | In-memory hash join |
-| `loop` | Nested loop join |
-
-### Join Example
+Use `expand` for relationship traversal — the live spelling for related records — or a
+dotted `fields` path (`'customer.name'`) for a single related column:
 
 ```typescript
 {
   object: 'order',
-  fields: ['id', 'total', 'customer.name', 'customer.email'],
-  joins: [
-    {
-      type: 'inner',
-      object: 'customer',
-      on: ['order.customer_id', '=', 'customer.id'],
-      strategy: 'auto'
-    }
-  ]
+  fields: ['id', 'total'],
+  expand: {
+    customer_id: { object: 'customer', fields: ['name', 'email'] }
+  }
 }
 ```
 
@@ -440,13 +420,7 @@ call in the SQL driver's query builder). Use `expand` for relationship traversal
   object: 'article',
   search: {
     query: 'kubernetes deployment',
-    fields: ['title', 'body', 'tags'],
-    fuzzy: true,
-    operator: 'and',
-    boost: { title: 2.0, body: 1.0 },
-    minScore: 0.5,
-    language: 'english',
-    highlight: true
+    fields: ['title', 'body', 'tags']
   }
 }
 ```
@@ -455,19 +429,20 @@ call in the SQL driver's query builder). Use `expand` for relationship traversal
 |:---|:---|:---|
 | `query` | `string` | Search text |
 | `fields` | `string[]` | Fields to search (optional — defaults to all searchable) |
-| `fuzzy` | `boolean` | Enable fuzzy matching for typo tolerance |
-| `operator` | `'and' \| 'or'` | How to combine search terms |
-| `boost` | `Record<string, number>` | Field relevance weights |
-| `minScore` | `number` | Minimum relevance score (0–1) |
-| `language` | `string` | Language for stemming/stopwords |
-| `highlight` | `boolean` | Return highlighted matches |
+| `fuzzy` | `boolean` | `[EXPERIMENTAL — not enforced]` Fuzzy matching for typo tolerance |
+| `operator` | `'and' \| 'or'` | `[EXPERIMENTAL — not enforced]` How to combine search terms |
+| `boost` | `Record<string, number>` | `[EXPERIMENTAL — not enforced]` Field relevance weights |
+| `minScore` | `number` | `[EXPERIMENTAL — not enforced]` Minimum relevance score (0–1) |
+| `language` | `string` | `[EXPERIMENTAL — not enforced]` Language for stemming/stopwords |
+| `highlight` | `boolean` | `[EXPERIMENTAL — not enforced]` Return highlighted matches |
 
 <Callout type="warn">
 Only `query` and `fields` are implemented. The engine expands `search` into a driver-agnostic
 `$and`-of-`$or`-of-`$contains` filter (ADR-0061) — `fuzzy`, `operator`, `boost`, `minScore`,
 `language`, and `highlight` are accepted by `QuerySchema` but read nowhere in
-`expandSearchToFilter()` / `normalizeSearch()`, so they have no effect. Multiple search terms
-are always AND-ed regardless of `operator`.
+`expandSearchToFilter()` / `normalizeSearch()`, so they have no effect; since #4286 their
+`.describe()` markers say so. Multiple search terms are always AND-ed regardless of
+`operator`.
 </Callout>
 
 ### Pinyin recall (Chinese deployments)
@@ -496,46 +471,36 @@ by `@objectstack/plugin-pinyin-search`) recomputes the column on demand.
 
 ---
 
-## Window Functions
+## Window Functions — removed from the request surface
 
 <Callout type="warn">
-`windowFunctions` is only reachable by calling the SQL driver's `findWithWindowFunctions()`
-method directly. `ObjectQL.find()` / `.aggregate()` and the `POST /api/v1/data/:object/query`
-route never call it, so sending `windowFunctions` through the standard client/REST query path
-has no effect.
+`query.windowFunctions` was **removed in `@objectstack/spec` 18** (#4286):
+`ObjectQL.find()` / `.aggregate()` and the `POST /api/v1/data/:object/query` route never
+routed it anywhere, so sending it had no effect. The key is tombstoned — a query carrying
+it fails to parse with the upgrade prescription — and the
+`WindowFunction`/`WindowSpec`/`WindowFunctionNode` exports left with it.
 </Callout>
 
-| Function | Description |
-|:---|:---|
-| `row_number` | Sequential row number within partition |
-| `rank` | Rank with gaps for ties |
-| `dense_rank` | Rank without gaps for ties |
-| `percent_rank` | Relative rank as percentage |
-| `lag` | Access previous row value |
-| `lead` | Access next row value |
-| `first_value` | First value in window |
-| `last_value` | Last value in window |
-| `sum` / `avg` / `count` / `min` / `max` | Running aggregations |
-
-### Window Function Example
+Window functions remain a **SQL-driver door**: `findWithWindowFunctions()`, callable
+directly on a SQL driver instance (it is not on the `IDataDriver` contract). Its input
+is the driver's own flat shape:
 
 ```typescript
-{
-  object: 'employee',
-  fields: ['name', 'department', 'salary'],
+const ranked = await sqlDriver.findWithWindowFunctions('employee', {
   windowFunctions: [
     {
       function: 'rank',
       alias: 'salary_rank',
-      over: {
-        partitionBy: ['department'],
-        orderBy: [{ field: 'salary', order: 'desc' }]
-      }
+      partitionBy: ['department'],
+      orderBy: [{ field: 'salary', order: 'desc' }]
     }
   ]
-}
+});
 ```
 
+For request-level analytics, use `aggregations` + `groupBy`, or model rankings in
+report/dashboard metadata.
+
 ---
 
 ## Distinct & Group By

content/docs/protocol/objectql/index.mdx

@@ -396,6 +396,6 @@ See [Security Protocol](/docs/protocol/objectql/security) for details.
   <Card
     title="Query Syntax"
     href="/docs/protocol/objectql/query-syntax"
-    description="Write database-agnostic queries with filters, joins, and aggregations"
+    description="Write database-agnostic queries with filters, expand, and aggregations"
   />
 </Cards>

content/docs/protocol/objectql/query-syntax.mdx

@@ -70,11 +70,9 @@ interface QueryAST {
   offset?: number;                         // Skip records (OFFSET)
   top?: number;                            // Alias for limit (OData compat)
   cursor?: Record<string, unknown>;        // Keyset pagination cursor
-  joins?: JoinNode[];                      // Explicit JOINs
   aggregations?: AggregationNode[];        // Aggregation functions
   groupBy?: GroupByNode[];                 // GROUP BY targets (string | object)
   having?: FilterCondition;                // HAVING clause
-  windowFunctions?: WindowFunctionNode[];  // Window functions (OVER)
   distinct?: boolean;                      // SELECT DISTINCT
   expand?: Record<string, QueryAST>;       // Recursive relation loading
 }
@@ -92,14 +90,22 @@ on the `find()` path:
 
 | Member | Status |
 |:-------|:-------|
-| `joins` | No driver reads it — there is no `query.joins` / `ast.joins` consumer anywhere in `packages/` |
 | `having` | Never read: `engine.aggregate()` forwards only `object`/`where`/`groupBy`/`aggregations`, and the in-memory fallback has no HAVING stage |
 | `cursor` | Accepted by `EngineQueryOptions`, but no driver implements keyset pagination |
 | `distinct` | Not applied by `find()`; the SQL and in-memory drivers expose a separate `distinct(object, field, filters?)` method instead |
-| `windowFunctions` | Not applied by `find()`; `SqlDriver.findWithWindowFunctions()` is a separate method that is not on the `IDataDriver` contract |
-| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | Parsed by the schema but ignored: only `query` and `fields` drive the expansion |
+| `aggregations[].filter` | `[EXPERIMENTAL — not enforced]` — a SQL `FILTER (WHERE …)` affordance neither the SQL builders nor the in-memory fallback applies |
+| `search.fuzzy` / `boost` / `operator` / `minScore` / `language` / `highlight` | `[EXPERIMENTAL — not enforced]` — only `query` and `fields` drive the expansion |
 
 `top` is the exception that *is* honored — the engine normalises it to `limit`.
+
+Two members left this table by being **removed** (#4286, ADR-0049 enforce-or-remove):
+`joins` and `windowFunctions` are tombstoned in `@objectstack/spec` 18 — a query
+carrying either fails to parse with the upgrade prescription, and authoring one is a
+`tsc` error. Related records are read through `expand`; window functions remain a
+SQL-driver door (`SqlDriver.findWithWindowFunctions()`). The inert members that
+*remain* declared (`having`, `cursor`, `distinct`, and the experimental flags above)
+are tracked in the liveness ledger (`packages/spec/liveness/query.json`) pending
+their #4286 dispositions.
 </Callout>
 
 ### Key Types
@@ -118,7 +124,7 @@ interface AggregationNode {
   field?: string;        // optional for COUNT(*)
   alias: string;         // result column alias
   distinct?: boolean;    // DISTINCT before aggregation — in-memory path only
-  filter?: FilterCondition; // FILTER WHERE clause — protocol only, never applied
+  filter?: FilterCondition; // [EXPERIMENTAL — not enforced] FILTER WHERE clause — never applied
 }
 
 // FieldNode — one entry of the select list. A field name, optionally dotted to
@@ -742,51 +748,43 @@ expansion emits a plain `$contains`, which `SqlDriver` compiles to a parameteris
 `LIKE '%…%'` with no case folding — so the dialect's own `LIKE`/collation rules decide —
 while the in-memory driver matches with a case-insensitive regex. Only `select` /
 `status` option *labels* are matched case-insensitively by the expansion itself.
-`fuzzy`, `boost`, `operator`, `minScore`, `language`, and `highlight` are accepted by
-the schema but ignored by the expansion.
-
-### Joins (protocol only)
+`fuzzy`, `boost`, `operator`, `minScore`, `language`, and `highlight` carry
+`[EXPERIMENTAL — not enforced]` markers (#4286): the schema accepts them, the
+expansion ignores them.
 
-`QuerySchema` defines a `joins` array, but **no driver reads it** — there is no
-`query.joins` consumer anywhere in `packages/`, so a query carrying `joins` silently
-runs as a single-table query. Use `expand` (§4) for relationship loading, or two
-queries joined in application code.
+### Joins — removed (#4286)
 
-For reference, the protocol shape is:
+`query.joins` was **removed in `@objectstack/spec` 18** (#4286, ADR-0049
+enforce-or-remove): no driver ever read it, so a query carrying `joins` silently ran
+as a single-table query. The key is tombstoned — authoring it is a `tsc` error, and a
+query that still carries it (even as an empty array) fails to parse with the upgrade
+prescription. The `JoinNode` / `JoinType` / `JoinStrategy` exports left with it.
 
-```typescript
-const query: QueryAST = {
-  object: 'order',
-  fields: ['id', 'amount'],
-  joins: [
-    {
-      type: 'inner',
-      object: 'customer',
-      alias: 'c',
-      on: { 'order.customer_id': { $eq: { $field: 'c.id' } } },
-    },
-  ],
-};
-```
+Use `expand` (§4) for relationship loading — the live spelling for related records —
+a dotted `fields` path (`'owner.name'`) for a single related column, or two queries
+joined in application code.
 
-**Join types:** `inner`, `left`, `right`, `full`
+### Window Functions — removed from the request surface (#4286)
 
-### Window Functions
+`query.windowFunctions` was **removed in `@objectstack/spec` 18** (#4286): `find()`
+never applied it, so every OVER clause it declared was silently dropped. The key is
+tombstoned, and the `WindowFunction` / `WindowSpec` / `WindowFunctionNode` exports
+left with it — they declared `field` / `over` / `frame` members that no executor ever
+read.
 
-`windowFunctions` on a `QueryAST` is ignored by `find()`. The SQL drivers implement
-window functions through a separate `findWithWindowFunctions()` method, which is not on
-the `IDataDriver` contract and is not surfaced by `IDataEngine`:
+Window functions remain a **SQL-driver door**: `findWithWindowFunctions()`, which is
+not on the `IDataDriver` contract and is not surfaced by `IDataEngine`. Its input is
+the driver's own flat shape — function name, alias, and optional flat
+`partitionBy` / `orderBy`:
 
 ```typescript
 const ranked = await driver.findWithWindowFunctions('order', {
   windowFunctions: [
     {
       function: 'row_number',
       alias: 'rank',
-      over: {
-        partitionBy: ['customer_id'],
-        orderBy: [{ field: 'amount', order: 'desc' }],
-      },
+      partitionBy: ['customer_id'],
+      orderBy: [{ field: 'amount', order: 'desc' }],
     },
   ],
 });
@@ -796,7 +794,9 @@ const ranked = await driver.findWithWindowFunctions('order', {
 ```
 
 That method always selects `*` plus the window columns — it does not honor a `fields`
-projection.
+projection, and niladic rendering means argument-taking functions (`LAG(field)`)
+emit without their argument. For request-level analytics use `aggregations` +
+`groupBy` (§5).
 
 ---