Skip to content

Commit 026761e

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/amazing-darwin-2z3fab
# Conflicts: # packages/spec/scripts/check-generated.ts
2 parents ce022ba + e59786e commit 026761e

70 files changed

Lines changed: 3060 additions & 323 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
---
2+
"@objectstack/service-storage": minor
3+
---
4+
5+
feat(storage): released field files enter collection on deployments that verified their file migration — ADR-0104 D3 wave 2 PR-5b (#3459)
6+
7+
The gated, final step of the file-as-reference sequence. On a deployment whose
8+
`adr-0104-file-references` flag is verified (`os migrate files-to-references
9+
--apply`, #3617), releasing a field file's ownership — clearing the field, or
10+
deleting the owning record — now also tombstones the file
11+
(`status='deleted'` + `deleted_at`), which starts the `sys_file` lifecycle's
12+
declared 30-day grace window and, at its end, hands the row to the reap sweep.
13+
Re-referencing the id inside the window revives it, exactly like re-attaching
14+
an attachment.
15+
16+
**The two halves ship together, deliberately.** The same change extends the
17+
reap guard's sweep-time re-verify beyond `sys_attachment` join rows to the
18+
ownership columns: a tombstoned file whose `ref_*` columns name a current
19+
owner (re-claimed in the window, or a release/claim race) is un-tombstoned and
20+
vetoed. Tombstoning released files without that re-verify would have turned
21+
every release into a *guaranteed* byte delete — the guard's old check consults
22+
a table that is always empty for field files. This pairing was the standing
23+
hard constraint on #3459, locked by regression tests on both halves.
24+
25+
**Nothing changes for a deployment that has not migrated.** Release keeps
26+
clearing the ownership columns only, and released files are retained forever.
27+
Every way of not knowing — no flag row, an unreadable table, an engine that
28+
cannot be asked — reads as "not verified": the gate fails closed, toward
29+
retention. And the guard re-reads the flag *fresh* at sweep time (not the
30+
release path's memoized read), so a later failing migration run — a database
31+
that has drifted — closes the gate for already-written tombstones too, without
32+
a restart. Attachments-scope collection is unchanged and needs no flag.
33+
34+
The irreversible moment is therefore per deployment: day 30 after *that*
35+
deployment verified its migration and released a file — never the upgrade
36+
itself.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
"@objectstack/objectql": minor
3+
---
4+
5+
feat(objectql): `engine.isFileReferencesMigrationVerified()` is public — one memoized flag read for both in-process consumers (#3459 PR-5b)
6+
7+
The memoized per-deployment read of the `adr-0104-file-references` migration
8+
flag was private to the engine's media value-shape enforcement. The storage
9+
service's release path now asks the same question — may a released field file
10+
be tombstoned? — so the method is public and the release hooks reach it as an
11+
optional duck-typed member (an older engine or a test fake reads as "not
12+
verified", failing closed). One read, one invalidation
13+
(`invalidateDataMigrationFlags()`), no way for the two consumers to see
14+
different answers.
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
---
22
---
33

4-
docs(adr): ADR-0104 addendum — evidence for the remaining #3438 strict flips (D1 references/structured JSON gate on a per-deployment `os migrate value-shapes` scan; fresh datastores attest both flags at creation; the D2 action-param flip rides the 18.0 major with an escape hatch) — releases nothing.
4+
docs(adr): ADR-0104 addendum — evidence for the remaining #3438 strict flips (D1 references/structured JSON gate on a per-deployment `os migrate value-shapes` scan, advertised on the 16→17 upgrade path; fresh datastores attest both flags at creation; D2 action params flip strict-by-default in 17.0, `OS_ACTION_PARAMS_STRICT_ENABLED` deleted before it reaches `latest`, `OS_ALLOW_LAX_ACTION_PARAMS` as the escape hatch) — releases nothing.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
---
3+
4+
docs(adr): revise the ADR-0104 2026-07-30 addendum — D2 action params flip strict-by-default in 17.0 (opt-in env var deleted before it reaches `latest`, `OS_ALLOW_LAX_ACTION_PARAMS` escape hatch); D1 non-media keeps the per-deployment scan gate, now advertised on the 16→17 upgrade path — releases nothing.
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/objectql": patch
4+
"@objectstack/rest": patch
5+
---
6+
7+
feat(spec,objectql,rest): publish the audit-provenance and import-coercion vocabularies (#3786, #4173)
8+
9+
Two more hand-copied lists retired the same way, each replaced by one spec
10+
export and derivation at every consumer.
11+
12+
**`AUDIT_PROVENANCE_FIELDS`** (`@objectstack/spec/data`, with the
13+
`AuditProvenanceField` type) — the four columns `applySystemFields` injects on
14+
every audit-tracked object: `created_at`, `created_by`, `updated_at`,
15+
`updated_by`. That four-name list existed in at least four copies across two
16+
repos: the registry's injection if-chain, the rule-validator's `preserveAudit`
17+
allowlist ("Kept in sync with the registry's auto-injected audit fields" — by
18+
nothing), and two objectui render surfaces. Now:
19+
20+
- the registry's injection is table-driven, keyed by the tuple with a
21+
`satisfies Record<AuditProvenanceField, …>` clause — a name added to the spec
22+
without a column definition (or vice versa) is a compile error, the
23+
`APPROVER_VALUE_BINDINGS` discipline;
24+
- the rule-validator's `AUDIT_TIMELINE_FIELDS` derives from the same tuple;
25+
- `FIELD_GROUP_SYSTEM_FIELDS`' audit prefix derives from it too — one
26+
declaration even inside the file that hosts both;
27+
- objectui's `AUDIT_FIELD_BY_ROLE` already pins itself by subset assertion and
28+
can import the tuple directly once this release is published.
29+
30+
Injection behaviour is byte-identical — a conformance test pins every injected
31+
column's shape against the pre-refactor definitions.
32+
33+
**`IMPORT_BOOLEAN_TRUE_TOKENS` / `IMPORT_BOOLEAN_FALSE_TOKENS` /
34+
`IMPORT_REFERENCE_TYPES`** (`@objectstack/spec/data`) — the `/import` coercion
35+
vocabulary #4173 asked for. The server's `import-coerce.ts` now derives its
36+
`BOOL_TRUE` / `BOOL_FALSE` / `REFERENCE_TYPES` from these instead of owning
37+
them privately, and objectui's Import Wizard preview — which re-checks the same
38+
contract client-side so a cell is flagged red exactly when the server would
39+
reject it — can retire its pinned-inventory mirror once this release is
40+
published (the retirement path is written in that file's own header).
41+
`IMPORT_REFERENCE_TYPES` ships with the legacy `'reference'` spelling included,
42+
retiring the `+ 'reference'` literal both ends carried separately. The tables'
43+
own discipline is tested: sets disjoint, every token pre-normalized
44+
(lower-case, trimmed), and the Chinese / check-mark spreadsheet-reality tokens
45+
pinned by name.
46+
47+
No behaviour change anywhere: every derived value is byte-identical to the
48+
literal it replaces.
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
---
3+
4+
Tooling only — no package changes, nothing to release.
5+
6+
The docs-drift mapper (`scripts/docs-audit/affected-docs.mjs`) derives a changed
7+
file's package root from the filesystem — the deepest ancestor directory with a
8+
`package.json` — instead of a regex that special-cased only `packages/plugins/*`.
9+
Under the old derivation the 30 packages nested under the other six container
10+
directories (`services/`, `connectors/`, `apps/`, `qa/`, `triggers/`, `adapters/`)
11+
collapsed into their container, whose missing `package.json` left the npm name
12+
unresolved and the npm-name matching arm dead: any doc that names
13+
`@objectstack/service-automation` but never the repo path was a guaranteed miss
14+
(#4162), the one direction this tool promises to avoid. No hardcoded container
15+
list replaces the special case — that would fail the same way again on container
16+
number eight (#3786's pattern).
17+
18+
- Verified against real history: on #4161's commit the drift comment attribution
19+
changes from the directory name `packages/services` to
20+
`@objectstack/service-automation`. A synthetic service-automation-only change
21+
goes from 6 docs — all belonging to *other* services, matched via the coarse
22+
`packages/services` path token, with `automation/flows.mdx` absent — to the 4
23+
right ones, `flows.mdx` first among them.
24+
- Second arm (from #4162's comment thread): `<packageRoot>/scripts/**` is
25+
build/verification tooling and no longer counts as an implementation change
26+
(#4183 flagged 106 docs for a diff whose only code change was a new check
27+
script). Kept narrow: `package.json` and `src/scripts/**` stay counted.
28+
Publication check done — no package ships runtime code from `scripts/`; three
29+
plugins publish a lone `i18n-extract.config.ts` only for lack of a `files`
30+
field. A scripts-only change now maps to 0 docs (was 106); the exclusion is
31+
reported in the summary and as `scriptFilesSkipped` in `--json`, never silent.
32+
- `--self-test` now pins the package-root derivation and the tooling-script
33+
classifier too (32 cases, hermetic via an injected fake tree), closing the
34+
guard-of-the-guard hole: the original self-test pinned only the test-file
35+
matcher, so this bug was invisible to it. Includes the two invariants from the
36+
issue: a container directory must never come out as a package root, and
37+
`packages/x/package.json` is implementation while `packages/x/scripts/y.ts`
38+
is not.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/platform-objects": patch
4+
"@objectstack/metadata-protocol": patch
5+
"@objectstack/driver-mongodb": patch
6+
---
7+
8+
fix(spec): five exported symbols resolved to `any` — type the recursive schemas and gate it in CI (#4171)
9+
10+
A recursive Zod schema needs an explicit annotation to break its circular
11+
inference, and five of them took the cheapest one available:
12+
13+
```ts
14+
export const NavigationItemSchema: z.ZodType<any> = z.lazy(() => …);
15+
export type NavigationItem = z.infer<typeof NavigationItemSchema>; // → any
16+
```
17+
18+
It compiles, it validates correctly at runtime, and it silently throws the type
19+
away. `NavigationItem`, `FormField`, `JoinNode` and `NormalizedFilter` were all
20+
`any` on the published surface, plus `FieldNodeSchema` — which had no exported
21+
type alias yet, so `z.infer<typeof FieldNodeSchema>` was `any` and
22+
`QueryAST['fields']` with it.
23+
24+
That is worse than a missing export. #4115 tells every consumer that a local
25+
declaration under a spec export's name must be replaced by a binding to the
26+
spec — and for these, obeying it **replaced a precise type with `any`**.
27+
objectui's `NavigationItem` is a 118-line documented interface (`recordId`
28+
template variables, `requiresObject` / `requiresService` capability gates,
29+
`filters` precedence); every key of it exists in the spec's version, so by every
30+
available signal it read as a redundant fork safe to delete. Deleting it swapped
31+
a fully-typed interface for `any`, with no compile error anywhere to say so.
32+
33+
It is hard to catch by inspection because `any` is mutually assignable with
34+
everything, so the natural "are these the same type?" check answers *yes* in both
35+
directions and recommends precisely the wrong action. Same failure family as
36+
#4075's `[key: string]: any` on `ActionDef`: a type that agrees with everything
37+
reads as agreement.
38+
39+
**Now annotated with the real type**, using the pattern `QueryAST` already
40+
follows in `data/query.zod.ts` — infer the non-recursive part, tie the recursive
41+
knot in the type, so the keys stay derived from the schema instead of being
42+
hand-maintained beside it:
43+
44+
```ts
45+
const BaseXSchema = z.object({ …every non-recursive key });
46+
export type X = z.infer<typeof BaseXSchema> & { children?: X[] };
47+
export const XSchema: z.ZodType<X> = z.lazy(() => BaseXSchema.extend({
48+
children: z.array(XSchema).optional(),
49+
}));
50+
```
51+
52+
`z.infer` now resolves to the type it should always have been: `NavigationItem`
53+
is the nine-branch discriminated union, `FormField` the 30-key form-field
54+
contract (with `visibleOn` absent by construction — ADR-0089 D2 folds it into
55+
`visibleWhen` at the boundary), `JoinNode` and the newly exported `FieldNode`
56+
the query AST nodes, `NormalizedFilter` the normalized filter AST. Runtime
57+
validation is unchanged: every schema parses exactly what it parsed before.
58+
59+
**What the types immediately caught**, none of it visible while they were `any`:
60+
61+
- `account.app.ts` set `defaultOpen` on three nav groups — a key the spec has
62+
never declared. It worked only because objectui's `NavigationRenderer` still
63+
falls back to that legacy alias. Fixed at the producer per Prime Directive
64+
#12: the canonical key is `expanded`.
65+
- The MongoDB driver built its projection with `projection[field] = 1` over
66+
`query.fields`, so a relationship `FieldNode` would have keyed the projection
67+
on `"[object Object]"`. It now reads the node's field name.
68+
- `setup.app.ts`, `studio.app.ts` and `setup-nav.contributions.ts` are annotated
69+
with the PARSED `App` / `NavigationContribution` types but omitted
70+
`.default()`ed keys (`expanded`, `target`), as did the form fields
71+
`metadata-protocol` synthesizes for `getUiView` (`span`). Each now states the
72+
default it was relying on, matching what the surrounding literals already do
73+
for `active` / `isDefault` / `collapsible` / `collapsed` / `columns`.
74+
75+
**Gated, not just fixed** (`check:exported-any`, wired into the required
76+
`TypeScript Type Check` job). `api-surface.json` records that an export *exists*
77+
and never what it *resolves to*, which is how these survived a whole major with
78+
every gate green. The new scan reads the built `.d.ts` a consumer's import
79+
actually resolves to and fails on any exported type that resolves to `any` — or
80+
any exported schema whose output is `any`, the root cause, and the only reason
81+
`FieldNodeSchema` was visible at all. Its `KNOWN_ANY` ledger is shrink-only and
82+
currently empty. It self-tests against the real zod first, so if the internals it
83+
reads are ever renamed the gate fails loudly instead of quietly passing
84+
everything forever.
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
"@objectstack/metadata-protocol": minor
3+
---
4+
5+
fix(metadata)!: a `$filter` array that is not a filter AST is rejected, not passed through (#4121)
6+
7+
`isFilterAST` was being read as a *conversion* gate: an array it refused was
8+
assigned to `options.where` unconverted, leaving each backend to make sense of a
9+
value the protocol had already decided it could not parse.
10+
11+
Item 2 of #3948, filed as error-locality work. The investigation found it is
12+
more than that.
13+
14+
**It closes the last silently-unfiltered shape.** #3948 made the drivers throw
15+
on a bare triple with an unknown operator and on any element that is neither a
16+
join keyword nor a condition array. What it could not reach is a lone `['and']`
17+
or `['or']`: the driver sets its join mode, matches no element, emits **no
18+
predicate**, and returns every row. `isFilterAST` refuses it (a logical node
19+
needs `length >= 2`), so it arrived as an opaque `where` and no driver-side
20+
check applied. That is now a 400.
21+
22+
**For every other shape this is not a narrowing.** driver-sql throws on all of
23+
them, driver-memory throws, driver-mongodb reaches its own parser and fails at
24+
the server. Rejecting at the protocol changes *which* error the caller sees, not
25+
*whether* there is one — and the message is in the request's own vocabulary
26+
(`unrecognised operator "not in"`, `element 1 is number`, plus the recognised
27+
operator list) rather than a driver's internal builder state.
28+
29+
Scoped narrowly, because the regression to fear is rejecting something valid:
30+
31+
- only `Array.isArray(filter)` values are in scope — a `where` **object** is
32+
untouched, including `$and`/`$or`/`$gte` shapes;
33+
- an empty `[]` is left alone: it means "no filter", and every path already
34+
treats it that way;
35+
- `isFilterAST` accepts nested arrays, so `[[a,'=',1],[b,'=',2]]` and
36+
`['and', […], ['or', …]]` keep converting. A naive "arrays are suspect" rule
37+
would have broken exactly those, which is why the accepted shapes are pinned
38+
by more tests than the rejected ones.
39+
40+
Errors carry `status: 400` and `code: 'INVALID_FILTER'`, matching the
41+
`UNSUPPORTED_QUERY_PARAM` convention alongside.
42+
43+
Verified: 12 new tests driving the real `findData` normalisation, not a
44+
re-implementation of its rule — six for shapes that must keep converting, six
45+
for shapes that must be rejected, including the exact message text. Reverting the
46+
change fails six of them. Full `@objectstack/metadata-protocol` suite: **122
47+
tests across 19 files**, green.
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
---
3+
4+
Showcase-only: a `bulk_actions` view on `showcase_task` demonstrating
5+
`bulkActions` (declared object actions fanned out over the selected records),
6+
plus the UI tour note distinguishing it from `bulkActionDefs`. Releases
7+
nothing — `@objectstack/example-showcase` is private.

0 commit comments

Comments
 (0)