Merge remote-tracking branch 'origin/main' into fix/analytics-objectql-read-scope-3597

Author: os-zhuang

.changeset/analytics-client-dispatcher-alignment.md

@@ -0,0 +1,32 @@
+---
+"@objectstack/client": patch
+"@objectstack/runtime": patch
+---
+
+fix(client,runtime): analytics.meta/explain now call routes that actually exist (#3584)
+
+The route audit (#3563) ledgered four dispatcher↔client shape mismatches.
+Re-verification showed the two analytics shapes the client spoke —
+`GET /analytics/meta/:cube` and `POST /analytics/explain` — were served by
+**nothing**: not the dispatcher, not `@objectstack/rest`, not
+`service-analytics`. Both methods 404ed against every deployment.
+
+- `analytics.meta(cube?)` — FROM `GET /analytics/meta/:cube` TO
+  `GET /analytics/meta[?cube=<name>]`. The cube argument is now optional; when
+  given, the dispatcher threads it into `AnalyticsService.getMeta(cubeName?)`,
+  which always supported the filter. Responses now use the dispatcher envelope
+  (`{ success, data }`).
+- `analytics.explain(payload)` — FROM `POST /analytics/explain` TO
+  `POST /analytics/sql` (the dispatcher's SQL dry-run route, backed by
+  `generateSql`). Method name unchanged.
+
+No migration is expected in practice: a method that unconditionally 404ed can
+have no working callers (none exist in objectstack or objectui). Anyone who
+had hand-rolled fetches against the imaginary shapes should switch to the
+routes above.
+
+The two storage rows from the same audit are deliberately NOT reshaped: the
+presigned/chunked protocol the SDK speaks is registered autonomously by
+`service-storage` on any http-server and stays canonical; the dispatcher's
+bare `POST /storage/upload` / `GET /storage/file/:id` are reclassified in the
+route ledger as a `server-only` low-level compat surface.

.changeset/apimethod-enum-shrink.md

@@ -6,9 +6,8 @@
 
 feat(spec)!: shrink the `ApiMethod` enum to the six primitives — legacy values are stripped at parse, never honored (#3543, P2 of #3391)
 
-**BREAKING** (ships as `minor` per the lockstep launch-window policy — every
-package versions together, so a `major` here would promote the entire
-monorepo; the `!` marker and this changeset are the breaking-change record):
+**BREAKING** (the `!` marker and this changeset are the breaking-change
+record; the train ships as the v17 major — see the `v17-rc-anchor` changeset):
 the authored `enable.apiMethods` enum is now exactly the six
 primitives (`get`, `list`, `create`, `update`, `delete`, `bulk`). The eight
 legacy values (`upsert`, `aggregate`, `history`, `search`, `restore`, `purge`,

.changeset/approval-attachment-descriptors.md

@@ -0,0 +1,29 @@
+---
+"@objectstack/spec": patch
+"@objectstack/plugin-approvals": patch
+---
+
+fix(approvals): return decision attachments as file values, not "[object Object]" (#3504)
+
+`sys_approval_action.attachments` is a `Field.file`, so the column **stores an
+opaque `sys_file` id** (ADR-0104 D3 — the stored form of every media field). The
+ObjectQL read path resolves that id into its expanded
+`{ id, name, size, mimeType, url }` form on the way out. But `rowFromAction`
+mapped the column with `.map(String)`, collapsing each expanded value to the
+literal string `"[object Object]"`. Every `listActions` consumer (the approval
+inbox timeline) then received garbage: the attachment chip had no filename and
+its id was `"[object Object]"`, so opening it 404'd.
+
+- `ApprovalActionRow.attachments` is now `ApprovalActionAttachment[]` — the
+  expanded file value plus its id, so a consumer can label and open an
+  attachment without needing read access to the system `sys_file` object (which
+  regular approvers do not have).
+- Three read forms are accepted: the expanded value (the normal case), a bare id
+  (nothing to expand it into — storage service absent, file not committed), and
+  a legacy inline blob written before file-as-reference (`file_id` /
+  `mime_type`), until the backfill converts it. The id test reuses the
+  platform's `isFileIdToken`, so this and the engine's read resolver cannot
+  disagree about what counts as an id.
+- The decision *input* (`ApprovalDecisionInput.attachments`) is unchanged — it
+  still takes fileId strings, which is also exactly what the column stores. Only
+  the read shape changed.

.changeset/file-access-delegate.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/spec": minor
+"@objectstack/service-storage": patch
+"@objectstack/plugin-approvals": patch
+---
+
+fix(storage): let an object delegate file-read authorization to its service
+
+Fixes a regression from the governed-download change (ADR-0104 D3 wave 2): a
+**legitimate approver could see a decision attachment's filename but got 403
+opening it**, found by driving app-showcase in a browser as a real non-admin
+approver.
+
+Cause: a field-owned file's download was authorized by testing whether the
+caller can READ the owning row. For an ordinary business object that is right —
+row readability *is* the access rule. For `sys_approval_action` it is the wrong
+authority: the audit table is deliberately closed to ordinary approver
+positions (`operation 'find' … is not permitted for positions [auditor,
+everyone]`), so the test denied the very approver the attachment was filed for.
+The approvals *service* has always had the real rule, which is why the timeline
+listing the attachment returned 200 while the bytes returned 403.
+
+An object may now name a service to answer the question instead:
+
+- `ObjectSchema.fileAccessDelegate` — a kernel service that authorizes
+  downloads of files owned by that object's media fields.
+- `IFileAccessDelegate.authorizeFileRead(recordId, context)` — the contract.
+- `sys_approval_action` declares `'approvals'`; `ApprovalService.authorizeFileRead`
+  reuses the *same* gate `listActions` applies (visibility of the parent
+  request) rather than inventing a second, looser rule for the bytes.
+
+**Fails closed**: a declared delegate that is missing or does not implement the
+method denies, rather than silently reverting to the raw read it was declared to
+replace. Objects without the declaration are unchanged.
+
+Verified in the browser against app-showcase, both sides of the gate: the
+approver now downloads the real PDF (200), and an anonymous request is still
+refused (401) — the anonymous capability URL the original change closed stays
+closed. A decision attachment ends up exactly as readable as the decision it
+hangs off: never more, and no longer less.

.changeset/filter-context-tokens-gate.md

@@ -0,0 +1,47 @@
+---
+"@objectstack/spec": patch
+"@objectstack/lint": patch
+"@objectstack/cli": patch
+---
+
+feat(spec,lint): freeze the `{current_user_id}` filter vocabulary and fail the build on unresolvable placeholders (#3574)
+
+A dashboard widget filtered on `{current_user}` rendered `0`. Not an error — a
+zero, indistinguishable from a metric that is legitimately empty, with nothing
+in the console or the server log. `service_dashboard.my_open_cases_by_priority`
+in the HotCRM template had shipped broken this way since the day it was
+written.
+
+The token had never been part of the contract. Date macros were frozen in
+`date-macros.zod.ts` with a spec vocabulary, a lint-usable predicate, and a
+single client resolver; `{current_user_id}` had only prose in an `app.zod.ts`
+JSDoc and three ad-hoc client implementations that each handled one surface's
+filter shape. Nothing could tell an author their token was wrong.
+
+- **`@objectstack/spec`** — new `data/context-tokens.zod.ts` freezing
+  `CONTEXT_TOKENS` (`current_user_id`, `current_org_id`) as the sibling of
+  `DATE_MACRO_TOKENS`, with `isContextToken` / `isKnownFilterToken` /
+  `classifyFilterToken` and a `CONTEXT_TOKEN_SUGGESTIONS` near-miss table. The
+  module documents what the tokens are *not*: presentation scope, never an
+  access boundary — that is RLS, which uses the unrelated `current_user.id`
+  expression root.
+- **`@objectstack/lint`** — new `validateFilterTokens` (rule
+  `filter-token-unknown`, severity `error`). It walks `filter` / `filters` /
+  `runtimeFilter` subtrees across dashboards, objects, views, reports,
+  datasets, pages and apps, and reports any placeholder that resolves in
+  neither vocabulary. It scans for filter *keys* rather than enumerating known
+  surfaces, so a new surface following the convention is covered the day it
+  ships — enumerating surfaces is how the dashboard was missed in the first
+  place. Navigation `recordId` / `params` are deliberately out of scope: they
+  resolve `AppContextSelector` ids, which are meaningless in a filter.
+- **`@objectstack/cli`** — the gate runs in `os validate` and `os compile`.
+
+It is an error rather than a warning because of who authors this metadata. An
+AI reads a query returning `0` as a correct answer and builds on it; its
+correction loop is author → validate → fix, so a diagnostic only reaches it if
+it can fail the build. The three spellings the suggestion table covers —
+`{current_user}`, `{user_id}`, `{organization_id}` — are each correct
+*somewhere else* in the platform, which is exactly why authors reach for them.
+
+Also fixes a `ViewSchema` JSDoc example that documented `{user_id}`, a token
+that resolves nowhere.

.changeset/import-sanitize-row-errors.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/rest": patch
+"@objectstack/plugin-auth": patch
+---
+
+fix(import): sanitize row errors — never leak raw SQL, map constraint failures to human wording (#3566)
+
+A failing import row surfaced the driver's raw error verbatim. When a write hit
+a DB constraint (e.g. `sys_user.phone_number` is `unique`), the query builder
+embeds the entire failing statement in `err.message`, and `toFailedResult`
+handed that straight back — so the importer saw ``insert into `sys_user`
+(...) values (...) - UNIQUE constraint failed: sys_user.phone_number``. That is
+both unreadable and an information disclosure of the schema.
+
+- `sanitizeRowError()` (import-runner) maps the common constraint failures —
+  SQLite / MySQL / Postgres `UNIQUE` and `NOT NULL` — to human wording
+  ("A record with this `<column>` already exists.", "`<column>` is required.")
+  and, as a backstop, never lets a message that still reads as a SQL statement
+  reach the client (it salvages the driver's trailing reason, or falls back to
+  a generic message). Already-friendly messages (e.g. better-auth's "User
+  already exists") pass through unchanged. Applies to every import path.
+- `isLikelyEmail` now rejects non-ASCII addresses, so an address like
+  `x@柴仟.com` fails the import **dry-run** pre-check instead of passing client
+  and dry-run validation only to be rejected by better-auth's strict ASCII
+  validator at real-import time.

.changeset/pre.json

@@ -0,0 +1,84 @@
+{
+  "mode": "pre",
+  "tag": "rc",
+  "initialVersions": {
+    "@objectstack/docs": "4.2.1",
+    "@objectstack/example-crm": "4.0.91",
+    "@objectstack/example-showcase": "0.3.13",
+    "@objectstack/example-todo": "4.0.91",
+    "@objectstack/example-embed-objectql": "0.0.31",
+    "@objectstack/hono": "16.1.0",
+    "@objectstack/account": "16.1.0",
+    "@objectstack/setup": "16.1.0",
+    "@objectstack/studio": "16.1.0",
+    "@objectstack/cli": "16.1.0",
+    "@objectstack/client": "16.1.0",
+    "@objectstack/client-react": "16.1.0",
+    "@objectstack/cloud-connection": "16.1.0",
+    "@objectstack/connector-mcp": "16.1.0",
+    "@objectstack/connector-openapi": "16.1.0",
+    "@objectstack/connector-rest": "16.1.0",
+    "@objectstack/connector-slack": "16.1.0",
+    "@objectstack/console": "16.1.0",
+    "@objectstack/core": "16.1.0",
+    "create-objectstack": "16.1.0",
+    "@objectstack/formula": "16.1.0",
+    "@objectstack/lint": "16.1.0",
+    "@objectstack/mcp": "16.1.0",
+    "@objectstack/metadata": "16.1.0",
+    "@objectstack/metadata-core": "16.1.0",
+    "@objectstack/metadata-fs": "16.1.0",
+    "@objectstack/metadata-protocol": "16.1.0",
+    "@objectstack/objectql": "16.1.0",
+    "@objectstack/observability": "16.1.0",
+    "@objectstack/platform-objects": "16.1.0",
+    "@objectstack/driver-memory": "16.1.0",
+    "@objectstack/driver-mongodb": "16.1.0",
+    "@objectstack/driver-sql": "16.1.0",
+    "@objectstack/driver-sqlite-wasm": "16.1.0",
+    "@objectstack/embedder-openai": "16.1.0",
+    "@objectstack/knowledge-memory": "16.1.0",
+    "@objectstack/knowledge-ragflow": "16.1.0",
+    "@objectstack/plugin-approvals": "16.1.0",
+    "@objectstack/plugin-audit": "16.1.0",
+    "@objectstack/plugin-auth": "16.1.0",
+    "@objectstack/plugin-dev": "16.1.0",
+    "@objectstack/plugin-email": "16.1.0",
+    "@objectstack/plugin-hono-server": "16.1.0",
+    "@objectstack/plugin-pinyin-search": "16.1.0",
+    "@objectstack/plugin-reports": "16.1.0",
+    "@objectstack/plugin-security": "16.1.0",
+    "@objectstack/plugin-sharing": "16.1.0",
+    "@objectstack/plugin-webhooks": "16.1.0",
+    "@objectstack/dogfood": "0.0.39",
+    "@objectstack/downstream-contract": "0.0.37",
+    "@objectstack/http-conformance": "0.0.5",
+    "@objectstack/rest": "16.1.0",
+    "@objectstack/runtime": "16.1.0",
+    "@objectstack/sdui-parser": "16.1.0",
+    "@objectstack/service-analytics": "16.1.0",
+    "@objectstack/service-automation": "16.1.0",
+    "@objectstack/service-cache": "16.1.0",
+    "@objectstack/service-cluster": "16.1.0",
+    "@objectstack/service-cluster-redis": "16.1.0",
+    "@objectstack/service-datasource": "16.1.0",
+    "@objectstack/service-i18n": "16.1.0",
+    "@objectstack/service-job": "16.1.0",
+    "@objectstack/service-knowledge": "16.1.0",
+    "@objectstack/service-messaging": "16.1.0",
+    "@objectstack/service-package": "16.1.0",
+    "@objectstack/service-queue": "16.1.0",
+    "@objectstack/service-realtime": "16.1.0",
+    "@objectstack/service-settings": "16.1.0",
+    "@objectstack/service-sms": "16.1.0",
+    "@objectstack/service-storage": "16.1.0",
+    "@objectstack/spec": "16.1.0",
+    "@objectstack/trigger-api": "16.1.0",
+    "@objectstack/trigger-record-change": "16.1.0",
+    "@objectstack/trigger-schedule": "16.1.0",
+    "@objectstack/types": "16.1.0",
+    "@objectstack/verify": "16.1.0",
+    "objectstack-vscode": "16.1.0"
+  },
+  "changesets": []
+}

.changeset/prune-dead-audit-config-cluster.md

@@ -0,0 +1,38 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec)!: remove the dead `AuditConfig` cluster from `@objectstack/spec/system` (#1878 recheck loose-end)
+
+The entire `system/audit.zod.ts` module — `AuditConfigSchema`,
+`AuditStorageConfigSchema`, `AuditRetentionPolicySchema`,
+`AuditEventFilterSchema`, `SuspiciousActivityRuleSchema`,
+`DEFAULT_SUSPICIOUS_ACTIVITY_RULES`, and the `AuditEvent` /
+`AuditEventActor` / `AuditEventTarget` / `AuditEventChange` /
+`AuditEventType` / `AuditEventSeverity` shape schemas (plus all their
+type exports) — is removed. Verified zero consumers repo-wide: the live
+audit path (`plugin-audit`) imports none of it, defines its own
+`sys_audit_log` row shape, and captures **unconditionally** via engine
+hooks, so `AuditConfigSchema.enabled: false` advertised a semantic
+(turning the compliance ledger off) the platform deliberately rejects.
+Same ADR-0056 D8 family as the earlier `compliance.zod` / `masking.zod` /
+`RLSAuditConfig` / `PolicySchema` removals: security/compliance-shaped
+config must never merely look live.
+
+**Migration — every dead knob maps to a live surface (or is deliberately
+not configurable):**
+
+| Removed (never enforced) | Live replacement |
+| --- | --- |
+| `AuditConfigSchema.enabled` | none — audit capture is **always on** (compliance ledger; `object.zod` `trackHistory` contract) |
+| `eventTypes` / `excludeEventTypes` / `minimumSeverity` / `AuditEventFilterSchema` | none today — if event filtering ships it lands as an `audit` **settings** namespace (ADR-0069 pattern), not app metadata |
+| which fields/objects are summarized + History tab UI | object-level + field-level **`trackHistory`** (live, enforced by plugin-audit) |
+| `AuditRetentionPolicySchema` / `storage` | object **`lifecycle`** `audit` category (retain → archive → delete) + per-org settings overrides (ADR-0057) |
+| `SuspiciousActivityRuleSchema` / `DEFAULT_SUSPICIOUS_ACTIVITY_RULES` | none — no detection engine exists; security monitoring is org-operations tooling, not app-package metadata |
+| `AuditEvent*` shape schemas | the `sys_audit_log` object definition in `plugin-audit` is the row-shape source of truth |
+
+No first-party, example, or downstream-contract code imported any of
+these symbols; `defineStack` never accepted an `audit` key, so no stack
+config changes. Docs page `references/system/audit.mdx` is removed by
+regeneration; the security-context module doc now marks audit alongside
+the previously removed compliance/masking subsystems.

.changeset/rest-route-ledger-audit-guard.md

@@ -0,0 +1,31 @@
+---
+"@objectstack/rest": patch
+"@objectstack/client": patch
+"@objectstack/runtime": patch
+---
+
+feat(rest): route audit tranche 2 — the REST surface gets its own ledger +
+conformance guard (#3587, follow-up to #3563)
+
+The dispatcher tranche closed its 27 gaps and guards them (#3569…#3579), but
+`@objectstack/rest` mounts a second, larger surface the client also reaches —
+89 routes, never audited. `rest-route-ledger.ts` now records a reviewed
+disposition for every one of them (38 sdk, 43 gap, 3 server-only, 3 public,
+2 mismatch), and the guard is real enumeration on both sources: RouteManager
+routes via the `getRoutes()` introspection seam, and the two
+RouteManager-bypassing registrars (`package-routes.ts`,
+`external-datasource-routes.ts`) via captured mock-server registrations — no
+pinned-by-hand list. The client half
+(`rest-route-ledger-coverage.test.ts`) verifies every claimed method exists;
+a 43-gap ratchet is wired into CI. Every guard direction was negative-tested.
+
+Notable dispositions the audit surfaced: `POST /api/v1/packages` is a
+publish/install shape collision between REST and the dispatcher (REST
+registers first and wins) — ledgered `mismatch`; the REST
+`GET /ui/view/:object/:type` path dialect is unreachable by the SDK's
+query-param dialect — ledgered `mismatch`; `service-storage` /
+`service-i18n` mount a third route surface outside `@objectstack/rest`,
+explicitly out of scope here and tracked under #3587.
+
+No behavior change — data + tests only, plus a scope-note refresh in the
+runtime ledger pointing at the new REST ledger.

.changeset/retire-default-dispatcher-routes.md

@@ -0,0 +1,27 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec)!: delete `DEFAULT_DISPATCHER_ROUTES` — the dead route table that
+underwrote a false compliance verdict (#3586, #3563 follow-up)
+
+The const was consumed by nothing in the runtime — only its own tests and
+`api-surface.json`. It listed dispatcher branches that never existed
+(`/workflow`, `/realtime`) while omitting eight real prefixes (`/keys`,
+`/mcp`, `/mcp/skill`, `/actions`, `/security`, `/share-links`, `/ready`,
+`/openapi.json`), and `CLIENT_SPEC_COMPLIANCE.md` anchored a "FULLY
+COMPLIANT" verdict on it while 27 real routes had no SDK expression.
+
+The audited, guard-enforced source of truth for the dispatcher's route
+surface is `packages/runtime/src/route-ledger.ts` (#3569): the conformance
+suite fails when the registry and the ledger drift, which the dead table
+never could.
+
+Also swept the last GraphQL fixture debris that #3562's surface removal
+left behind: registry test fixtures renamed to honest OData naming, the
+tautological `config.graphql` assertions dropped, and the stale
+`"type": "graphql"` JSDoc example in `registry.zod.ts` corrected.
+
+Breaking for anyone importing `DEFAULT_DISPATCHER_ROUTES` (a repo-wide and
+objectui-wide grep shows zero consumers); shipped as minor per the
+launch-window convention, cf. #3562/#3581.