Merge remote-tracking branch 'origin/main' into claude/app-metadata-validation-assessment-xskbcg

# Conflicts:
#	packages/spec/api-surface.json

Author: claude

.changeset/adr-0104-deployment-migration-gate.md

@@ -0,0 +1,58 @@
+---
+"@objectstack/spec": minor
+"@objectstack/platform-objects": minor
+"@objectstack/service-storage": minor
+"@objectstack/objectql": minor
+"@objectstack/cli": minor
+---
+
+feat(migrate): `os migrate files-to-references` — a data migration with a self-check, gated per deployment (#3617)
+
+The ADR-0104 file-as-reference migration ships as a command a deployment runs
+against its own database, and the deployment-level flag it records is what may
+later authorise irreversible behaviour — never the platform version.
+
+```bash
+os migrate files-to-references           # dry run: reports, writes nothing
+os migrate files-to-references --apply   # converts, verifies, records the flag
+```
+
+The run backfills legacy file-field values (inline metadata blobs, own-resolver
+URLs, `data:` URIs) into owned `sys_file` references, reconciles the ownership
+ledger against what records actually hold, and — only on an `--apply` run whose
+reconciliation reports **zero blocking discrepancies** — records
+`sys_migration { id: 'adr-0104-file-references', verified_at, blocking: 0 }`.
+
+**Why a flag rather than a release note.** ObjectStack is a development
+platform: third-party deployments upgrade on their own schedule and their data
+is not observable by anyone else, so no release-side soak can vouch for them.
+The evidence has to be produced where the data is. Consequences:
+
+- Installing a new version never starts deleting bytes. Running the migration
+  and passing its self-check is the consent.
+- Not run, or not passed → files are retained forever. Wasted storage, zero
+  data loss.
+- A later failing run **clears** `verified_at`: a deployment whose data has
+  drifted closes its own gate.
+- A dry run writes nothing at all — not the conversions, and not the flag,
+  even when the self-check would pass.
+- External URLs stay advisory. They are not `sys_file`s, so they can never
+  enter collection; whether to remodel them as a `url` field is the app
+  author's decision (ADR-0104 R7), not a gate.
+
+Ships alongside:
+
+- `@objectstack/spec` — `DataMigrationFlagSchema`, `FILE_REFERENCES_MIGRATION_ID`,
+  and the single `isDataMigrationFlagVerified` predicate both future consumers
+  (collection #3459, strict value-shape #3438) read, so the two gates cannot
+  disagree about the same fact.
+- `@objectstack/platform-objects` — the `sys_migration` object plus
+  `readDataMigrationFlag` / `isDataMigrationVerified` / `recordDataMigrationRun`.
+  Reads fail toward "not verified": a gate that cannot read its evidence stays
+  closed.
+- `@objectstack/objectql` — a read may now opt out of file-reference expansion
+  via the spec's `RAW_FILE_VALUES_CONTEXT_KEY`, and the storage service's
+  bookkeeping/scan reads do. Without it the read resolver rewrites stored ids to
+  their expanded form before the reconciliation sees them, which reports held
+  references as absent — noisy `stale_owner` findings, and a missed
+  `unowned_reference` would have been a false pass of the collection gate.

.changeset/analytics-label-read-scope.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/service-analytics": patch
+---
+
+fix(analytics): scope the dimension-label lookup to the referenced object's RLS (#3602)
+
+When a dataset groups by a `lookup`/`master_detail` dimension, analytics resolves
+the grouped FK ids to the related record's display name via a per-record read
+(`group by id`) dressed as an aggregate. That read carried **no read scope**, so
+it revealed related-record display names whenever the referenced object's RLS is
+stricter than the base object whose rows carry the id — a user could see a name
+the referenced object's own RLS would hide. (Same-object and looser-referenced
+cases were already safe because the ids come from the post-#3597 scoped
+aggregate; this closes the stricter-referenced case.)
+
+The label lookup now applies the **referenced object's own** read scope — bound
+to the request via the same `getReadScope` provider the aggregate path uses,
+composed with `$and` (never key-merge) so it can't be displaced by the id
+predicate. Fail-closed: if that object's scope can't be resolved, the dimension's
+labels are skipped (the raw id renders) rather than fetched unscoped. No behaviour
+change when no read-scope provider is configured.
+
+Internal `DimensionLabelDeps.fetchRecordLabels` gains an optional `scope` argument
+and `resolveDimensionLabels` an optional `resolveScope` resolver; both are
+service-analytics-internal (no spec/contract change).

.changeset/analytics-widget-query-options.md

@@ -0,0 +1,53 @@
+---
+"@objectstack/spec": patch
+"@objectstack/service-analytics": patch
+"@objectstack/rest": patch
+---
+
+feat(analytics): honour widget `dateGranularity`, `sortBy`/`sortOrder`, and `limit` in the dataset query (#3588)
+
+Three presentation options were accepted by the metadata layer and then dropped
+by the analytics query builder. They reached no SQL, produced no error, and the
+only way to notice was to read the `sql` a dataset response echoes — so a
+dashboard could declare `dateGranularity: 'month'` and quietly render one bar
+per record.
+
+- **`dateGranularity` now buckets.** `DatasetSelection` gained an optional
+  `dateGranularity`, applied to every selected `date` dimension. Precedence per
+  dimension: an explicit `timeDimensions` granularity, then the selection's,
+  then the dataset dimension's own default. A widget can bucket a trend by month
+  without the dataset committing every other consumer to that granularity.
+- **`order` / `limit` / `offset` now apply on every path.** They are applied to
+  the ASSEMBLED grid — after measure-scoped sub-queries merge, after `compareTo`
+  columns attach, and after derived measures are computed — so a derived measure
+  is a valid sort key and the ObjectQL aggregate path (which has no ordering
+  grammar, and which native SQL hands every date-bucketed query to) orders
+  identically to native SQL. A single-query selection still pushes the window
+  down into the statement. An `order` key that names nothing the selection
+  projects is now rejected (400) rather than silently ignored.
+- **`limit` is deterministic.** Without an `order`, a limit orders by the
+  selected dimensions first, so it truncates a reproducible window instead of an
+  arbitrary subset.
+- **Widget `options` is a contract again.** The four query-affecting keys
+  (`dateGranularity`, `sortBy`, `sortOrder`, `limit`) plus `stageOrder` are
+  declared on `DashboardWidgetOptionsSchema`, so a typo like `sortDirection` is
+  an author-time error. The bag stays open — renderer extras (`icon`, `columns`,
+  `striped`, …) pass through untouched.
+
+Two latent bugs surfaced while fixing the above and are fixed here too:
+
+- `order`/`limit` were forwarded to EVERY sub-query. A measure-scoped
+  supplementary query selects one measure, so an inherited `ORDER BY` named a
+  column it never selected, and an inherited `LIMIT` truncated it before the
+  merge — dropping rows from the assembled grid. Nothing hit this only because
+  nothing passed `order`.
+- The `compareTo` pass built its query by hand and skipped granularity
+  resolution, so a month-bucketed primary grid was merged against raw-timestamp
+  comparison rows. No dimension key matched and every `<measure>__compare`
+  column came back empty.
+
+`ObjectQLStrategy` now also echoes a representative `sql` (with `date_trunc`,
+`WHERE`, `ORDER BY`, and `LIMIT`; filter values parameterized, never inlined).
+Previously the `sql` field simply vanished from the response whenever a query
+was date-bucketed, leaving an author unable to tell "not implemented" from "this
+strategy doesn't report".

.changeset/better-auth-team-member-count.md

@@ -0,0 +1,43 @@
+---
+"@objectstack/platform-objects": patch
+"@objectstack/plugin-auth": patch
+---
+
+fix(auth): provision the better-auth 1.7 columns `sys_team` / `sys_team_member` / `sys_two_factor` were missing (#3624)
+
+better-auth 1.7.0-rc.1 added fields to three models that the platform objects
+never provisioned and `auth-schema-config.ts` never mapped. Because an unmapped
+field keeps its camelCase name, the adapter emitted columns no table had:
+
+| model | field | column now provisioned |
+|:---|:---|:---|
+| `team` | `memberCount` | `sys_team.member_count` |
+| `teamMember` | `membershipKey` | `sys_team_member.membership_key` |
+| `twoFactor` | `failedVerificationCount` / `lockedUntil` | `sys_two_factor.failed_verification_count` / `locked_until` |
+
+The team pair broke org creation outright. The organization plugin's team
+sub-feature is on by default, so `POST /api/v1/auth/organization/create`
+auto-creates a default team — and that insert died with `table sys_team has no
+column named memberCount` *after* the organization row had already committed.
+Callers got an HTTP 500 on top of a half-created org: a real org row with no
+default team behind it. Every multi-org deployment's create-org flow hit this.
+
+The two-factor pair broke the 2FA lockout path the same way: better-auth
+guard-increments `failedVerificationCount` on each wrong code and stamps
+`lockedUntil` past the threshold, so a wrong code 500'd instead of being
+counted. All four columns are better-auth's own state — provisioned, readable,
+and never written from the ObjectStack side.
+
+Existing environments pick the columns up through the driver's additive schema
+sync; no data migration is needed. `member_count` backfills to 0 and
+better-auth's own `syncTeamMemberCount` reconciles it on the next membership
+change, and `membership_key` stays null on pre-upgrade rows, which better-auth
+tolerates by falling back to the `(team_id, user_id)` pair.
+
+A new drift gate (`better-auth-schema-parity.test.ts`) now asserts that every
+column the installed better-auth version can write exists on the platform
+object backing it, across the auth manager's whole model surface. The ADR-0092
+D7 guard only ever caught *collisions* between our extension fields and
+better-auth's, so a bump that adds a brand-new field passed the build and failed
+at runtime — twice now, counting the 1.7 `oauthAccessToken.authorizationCodeId`
+regression. The next one fails the build instead.

.changeset/client-url-conformance-capstone.md

@@ -0,0 +1,41 @@
+---
+"@objectstack/client": patch
+---
+
+test(client): close the route audit's reverse direction — every SDK URL must match a route some surface mounts (#3642)
+
+The capstone of the #3563 route audit. The dispatcher (#3563), REST (#3587) and
+service-mount (#3636) ledgers all run server → client: enumerate what a surface
+mounts, demand a reviewed disposition, and for `sdk` rows demand the named
+client method exists. None of them asked the reverse question — does the URL
+the client *builds* match anything a server *mounts*? — so a method could name
+a real function, carry a green ledger row, and 404 everywhere.
+
+That shipped four times, found one at a time by hand: `analytics.explain` and
+`analytics.meta` (#3584), `meta.getView` (#3611), and `i18n.getTranslations` /
+`getFieldLabels` (#3636) — the last pair having carried green `sdk` rows since
+tranche 1.
+
+`client-url-conformance.test.ts` drives every method on a real client with a
+recording `fetch` and matches each captured URL against the union of all four
+ledgers. A real drive rather than a hand-written "method X targets route Y"
+table, because such a table is an assertion *about* the code that the code can
+drift away from — the exact failure being fixed. Mutation-checked: re-injecting
+the #3636 dialect bug fails the suite.
+
+The sweep's own completeness is asserted, since that is what rots silently — a
+new method must be driven or declared `NON_HTTP` with a reason; a driven method
+emitting zero requests fails (stale placeholder args are how a sweep quietly
+stops covering anything); a URL containing `undefined` fails; and the
+`__api-endpoint` `(unmatched)` catch-all is excluded from the pattern set so it
+cannot match everything and make the suite vacuous.
+
+196 of ~219 methods matched. Two bounds are reported rather than papered over:
+`/api/v1/cloud/*` (23 `projects.*` methods) belongs to the sibling `cloud` repo
+and is exempt by prefix, bounded so no other namespace can use it (#3655); and
+60 of ~196 matched calls rest only on a `**` prefix claim rather than a
+resolvable route — 54 of those on `* /auth/**` — a count the guard ratchets so
+it can only shrink (#3656).
+
+No runtime change: this is a guard plus the ledger-header and audit-doc notes
+recording what it does and does not cover.

.changeset/export-empty-result-header.md

@@ -0,0 +1,19 @@
+---
+"@objectstack/rest": patch
+---
+
+fix(rest): export emits the projected header row on an empty result set (#3547)
+
+`GET /data/:object/export` wrote a zero-byte file whenever the query matched no
+rows — the header was only ever written alongside the first data chunk. With the
+`getReadableFields` column projection the readable column set is derived from
+schema + context, so it is known even when no rows come back: an empty CSV/xlsx
+export now carries the exact readable header, which also makes it a usable
+import template.
+
+The header is emitted only when the column set is AUTHORITATIVE — the security
+service's readable projection, or an explicit `?fields=` request. When the header
+is schema-derived and the projection was unavailable, the export stays headerless
+as before: the masked-row fallback has no rows to narrow with, and writing the
+full schema header would name FLS-hidden columns. `header=false` still suppresses
+the header in every case.

.changeset/invitation-accepted-host-seam.md

@@ -0,0 +1,20 @@
+---
+"@objectstack/plugin-auth": minor
+---
+
+feat(auth): `onInvitationAccepted` host seam — better-auth's
+`afterAcceptInvitation` forwarded to the host (ADR-0105 D8 prerequisite)
+
+An invitation may carry placement intent (target business unit + positions,
+extension fields on `sys_invitation` per the ADR-0092 whitelist), but there
+was no server-side seam to apply it when the invitation is accepted —
+better-auth's org-plugin models don't fire core `databaseHooks` (framework
+#3541 D8 note).
+
+`AuthManagerConfig.onInvitationAccepted` mirrors `onOrganizationCreated`:
+invoked from `organizationHooks.afterAcceptInvitation` with the mapped ids
+(`invitationId`, `organizationId`, `userId`, `memberId`, `role`, `email`)
+plus the RAW `invitation` / `member` rows so a host reads its own extension
+columns without a second query. Failure-isolated — acceptance never rolls
+back on a side-effect miss; hosts needing effectively-atomic placement
+should make the callback idempotent and reconcile on retry.

.changeset/page-header-i18n-3589.md

@@ -0,0 +1,39 @@
+---
+"@objectstack/spec": minor
+"@objectstack/platform-objects": minor
+"@objectstack/cli": minor
+"@objectstack/rest": patch
+---
+
+feat(spec): resolve page metadata i18n — `page:header` title/subtitle (#3589)
+
+Custom system pages authored as metadata (Installed Apps, Cloud Connection,
+Connect an Agent) hard-code their `page:header` copy in
+`properties.title` / `properties.subtitle`. Every other metadata type is
+localized at the REST boundary, but `page` was not: the `pages` namespace
+existed only on `AppTranslationBundleSchema` — a schema no runtime reads —
+with no resolver behind it, so those headers stayed English in every locale
+while the matching nav labels translated correctly.
+
+- `TranslationDataSchema` (the shape the i18n service actually serves) gains a
+  `pages` namespace: `pages.<name>.{label,description,title,subtitle}`.
+- New `translatePage` in `@objectstack/spec/system` translates a page's own
+  `label` / `description` and overlays `title` / `subtitle` onto every
+  `page:header` in the page's regions. Registered in
+  `translateMetadataDocument`, so it rides the existing read path.
+- `page` added to the REST boundary's `TRANSLATABLE_META_TYPES`. Locale
+  extraction, the locale-keyed ETag, and `Vary: Accept-Language` already
+  covered every metadata type — no new plumbing.
+- `objectstack i18n extract` now emits page entries, including the
+  `page:header` copy, so the new namespace is not invisible to the tooling.
+- zh-CN / ja-JP / es-ES translations shipped for the three Setup pages, plus
+  the missing `nav_cloud_connection` / `nav_connect_agent` nav labels (these
+  existed only in zh-CN).
+
+Header copy is keyed by **page name**, not by component id: `page:header`
+instances carry no stable id. `title` falls back to `pages.<name>.label`, since
+a page's header title and its nav label are normally the same string.
+
+Authoring is unchanged and English literals stay in metadata as the fallback —
+a page with no `pages` entry renders exactly as before. Consumers of
+`@object-ui` need no change: pages arrive already localized from the server.

.changeset/route-audit-tranche-3-service-mounts.md

@@ -0,0 +1,51 @@
+---
+"@objectstack/service-storage": patch
+"@objectstack/service-i18n": patch
+"@objectstack/client": patch
+---
+
+fix(client,service-i18n): ledger the autonomously-mounted service routes, and repair the two i18n calls that reached nothing (#3636)
+
+Tranche 3 of the #3563 route audit — the last un-audited server surface. The
+dispatcher ledger (#3563) and the REST ledger (#3587) each stop at their own
+package boundary, and two services mount routes outside both: they reach for
+the `http-server` service and register straight on `IHttpServer`, so neither
+`RouteManager` nor `RestServer.getRoutes()` has ever seen them. That left the
+SDK's entire storage surface, plus all of i18n, in the pre-#3563 posture:
+expressed, working, guarded by nothing.
+
+**Ledgers + guards.** `storage-route-ledger.ts` (10 routes) and
+`i18n-route-ledger.ts` (3) sit next to the registrars that mount them, each
+enumerated for real — the registrar runs against a capturing mock
+`IHttpServer` and its registration calls *are* the route set, so a new route
+lands with a reviewed disposition or fails CI. The client half is
+`packages/client/src/service-route-ledger-coverage.test.ts`; ledgers cross the
+boundary as relative source imports, never a service→client package edge.
+
+**Two wire-level 404s fixed.** `i18n.getTranslations` sent
+`/i18n/translations?locale=xx` and `i18n.getFieldLabels` sent
+`/i18n/labels/:object?locale=xx`, while every serving surface — service-i18n's
+mounts, the dispatcher's HTTP mounts, and the `plugin-rest-api.zod.ts`
+contract — mounts only the path form. Neither call could ever be answered.
+Both had carried a green `sdk` row in the dispatcher ledger since tranche 1,
+because that guard asks whether the client *method* exists, not whether it
+speaks a URL anything mounts. The client now sends the path dialect, the same
+resolution #3611 gave `meta.getView`, and a new suite drives the real client
+at a real router so a revert cannot pass quietly.
+
+**One response-shape fix.** service-i18n's success bodies omitted the
+`success` flag that `ObjectStackClient.unwrapResponse` keys on, so the SDK
+returned the raw `{ data: … }` wrapper against that provider while returning
+the declared unwrapped shape against the dispatcher — one method, two shapes,
+decided by which plugin mounted the route. Its three handlers now emit the
+`{ success: true, data }` envelope the `i18n` route group declares. `data` did
+not move, so direct body readers are unaffected.
+
+Storage audited clean: 7 routes SDK-expressed, 3 reviewed `server-only` (the
+browser capability URL objectql stamps into file-field payloads, and the two
+local-driver loopbacks). The chunked-upload family, flagged for triage, turned
+out fully expressed. Both ledgers ratchet `gap` and `mismatch` at zero.
+
+Filed, not fixed: `GET {base}/_local/file/:key` is built by three call sites
+and mounted by none (#3641); the cross-surface URL conformance guard that would
+have caught all of the above mechanically is the capstone (#3642).