Skip to content

Commit f860ae5

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/app-metadata-validation-midr20
# Conflicts: # scripts/i18n-coverage-baseline.json
2 parents 487c020 + 4921a95 commit f860ae5

61 files changed

Lines changed: 2010 additions & 297 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
---
2+
"@objectstack/driver-sql": patch
3+
"@objectstack/service-analytics": patch
4+
---
5+
6+
fix(driver-sql,analytics): stop `aggregate()` / `distinct()` leaking SQLite's raw epoch storage (#3797)
7+
8+
Both returned `await builder` directly, without the `formatOutput` pass every
9+
`find()` row gets. On SQLite — the one dialect where a `Field.datetime` is
10+
stored as INTEGER epoch milliseconds rather than a native timestamp — that raw
11+
storage form went straight to the caller:
12+
13+
| call | before | after |
14+
| --- | --- | --- |
15+
| `find()` | `"2026-01-10T09:00:00.000Z"` | unchanged |
16+
| `distinct('closed_at')` | `[1768035600000]` | `["2026-01-10T09:00:00.000Z"]` |
17+
| `aggregate()` `max(closed_at)` | `1768035600000` | `"2026-01-10T09:00:00.000Z"` |
18+
| `aggregate()` `groupBy: ['closed_at']` | key `1768035600000` | key `"2026-01-10T09:00:00.000Z"` |
19+
20+
Same root cause as #3773, different exit. `Field.date` was never affected — it
21+
is ISO TEXT on every dialect, so its storage form already equals its
22+
presentation.
23+
24+
The visible surfaces were a `_max`/`_min` measure over a datetime (a "last
25+
closed" KPI tile rendered `1768035600000`) and a `groupBy` on a raw datetime
26+
dimension, which also disagreed with the in-memory `applyInMemoryAggregation`
27+
fallback — that one consumes already-formatted `find()` rows, so the same
28+
dataset changed key type depending on which path served it.
29+
30+
Which columns hold an instant is now recorded while the statement is built,
31+
because that is the only point where a column name and its meaning are both
32+
known: a `min()` lands under its alias and never under the field name, while a
33+
date-BUCKETED column lands under the field name but holds a label (`'2026-01'`)
34+
rather than an instant. Matching on names afterwards gets both backwards.
35+
36+
`distinct()` additionally re-deduplicates after presenting: SQL `DISTINCT`
37+
compares STORED values, and one SQLite datetime column holds both INTEGER and
38+
TEXT forms, so two rows recording the same instant survived as two and then
39+
presented identically. It has no in-repo callers today; this keeps it honest
40+
rather than leaving a second convention in the driver.
41+
42+
**`cross-object-rebucket` was fixed alongside it, because presenting min/max
43+
correctly is what exposed it.** `recombine()` coerced every operand with
44+
`Number()`, which silently depended on receiving an epoch: handed the ISO string
45+
the driver now returns it produced `NaN`, and on Postgres/MySQL (where knex
46+
returns a `Date`) it had always flattened the value back to an epoch integer one
47+
layer above the driver. `min`/`max` now order by the instant and return the
48+
winning value in the shape it arrived in; `sum`/`count` stay numeric.
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
---
3+
4+
test(service-storage): give the attachment read-visibility harness real Filter Protocol semantics
5+
6+
Test-only — releases nothing.
7+
8+
`attachment-read-visibility.test.ts` faked its engine with a matcher that
9+
understood implicit equality and `$in` and nothing else: no `$and`, no `$or`,
10+
no `$not`. Every assertion in the file could therefore only check the *shape*
11+
of the filter `computeParentVisibilityFilter` emits, never the rows that filter
12+
selects — a poor bargain for a predicate whose whole job is narrowing.
13+
14+
Changes:
15+
16+
- The harness matcher now implements the protocol for real, mirroring
17+
`driver-memory/memory-matcher.ts` and `formula/matches-filter.ts`: every key
18+
in a filter object ANDs, `$or` ORs its branches, and a branch's own contents
19+
still AND. Operators it does not implement now **throw** instead of being
20+
ignored — a silently under-implemented test double is the failure mode this
21+
change exists to close.
22+
- A new case asserts the **rows** a multi-parent-type scope returns, not just
23+
its shape: rows whose discriminator matches a branch but whose id is absent
24+
from that branch's id list (including one that borrows the sibling branch's
25+
id) are excluded, so the per-branch pairing itself is pinned.
26+
- A conformance block pins the harness matcher against the same 2x2 fixture and
27+
expectations as `memory-matcher-or-semantics.test.ts`,
28+
`matches-filter-or-semantics.test.ts` and `sql-driver-or-filter.test.ts`, so
29+
the four cannot drift apart, plus a case proving the matcher actually
30+
distinguishes a correctly paired scope from a widened one.
Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
---
2+
"@objectstack/cli": patch
3+
---
4+
5+
fix(cli): finish the `--json` truncation fix — every command, and the second document it was hiding (#3780 follow-up)
6+
7+
#3780 routed three commands through `emitJson`. The other ~100 emission sites
8+
still wrote machine output with `console.log`, which on a **pipe** is cut off
9+
at one 64 KiB buffer when the command exits right after: Node buffers pipe
10+
writes asynchronously and the exit tears the process down mid-drain. It is
11+
invisible to whoever writes it — stdout to a TTY is synchronous, so every
12+
interactive run looks perfect while every scripted consumer, the only audience
13+
`--json` has, gets invalid JSON.
14+
15+
The exit does not have to be an explicit `process.exit`. oclif ends failing
16+
commands with `handle()``Exit.exit()``process.exit()` and flushes
17+
nothing on that path (`flush()` runs only on `execute()`'s success path), so a
18+
plain `this.exit(1)` — or any thrown error — truncates identically. 73 of the
19+
104 sites had exactly that shape. Even `lint` was only half fixed: `--eval
20+
--json` writes a whole corpus report and was still on `console.log`.
21+
22+
All 104 sites now go through `emitJson`, `formatOutput`'s `json`/`yaml`
23+
branches through the same drain-aware write (`--format yaml` truncated too),
24+
and an ESLint rule keeps the pattern from growing back one command at a time.
25+
Control flow is untouched — a following `this.exit(1)` stays, it is simply
26+
safe once the buffer has drained. Output bytes are unchanged: roughly half the
27+
sites emitted compact and half indented, and each keeps whichever it had.
28+
29+
**Draining the write exposed a second defect underneath it.** Because
30+
`this.exit(1)` *throws*, a command whose body sits in one `try` unwinds its
31+
inner "report and stop" into the outer `catch`, which reports again — so
32+
`os validate --json` on a failing config printed **two** JSON documents, which
33+
is neither valid JSON nor valid JSONL. Truncation had been hiding the second
34+
one. Nine commands had this shape; their catch clauses now re-throw the exit
35+
signal (`isExitSignal`) instead of describing it as a failure.
36+
37+
Measured on `os validate --json` against a config with 900 schema errors,
38+
piped:
39+
40+
| | bytes | parses |
41+
|---|---:|---|
42+
| before | 131072 (exactly two buffers, cut mid-string) | no |
43+
| truncation fix alone | 1514711 (two documents) | no |
44+
| both fixes | 1514648 (one document, 900 errors) | **yes** |
45+
46+
Pinned end to end: a real command, a real pipe, a payload past several
47+
buffers — plus a control case asserting the `console.log` pattern it replaced
48+
genuinely truncates, so the gate cannot quietly stop testing anything.
49+
50+
Not covered: the ~30 human-facing `console.log` paths, which are unaffected,
51+
and `os serve` / `os dev` logging. This is deliberately not fixed by forcing
52+
stdout into blocking mode process-wide, which would be one line and cover
53+
everything — the same binary runs the dev server, and a blocking write to a
54+
pipe with a slow reader blocks the event loop, trading truncated JSON for a
55+
server that stalls on its own logs.
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
"@objectstack/platform-objects": patch
3+
---
4+
5+
fix(i18n): platform-objects' 231 untranslated strings were 1 — close the real gap and stop the phantom (#3762)
6+
7+
Closes the rest of #3762. The remaining item was recorded as "platform-objects
8+
is 77 strings short per locale, in `apps.*` / `dashboards.*`, and its
9+
`--objects-only` extract cannot scaffold them — needs an emit decision (drop
10+
`--objects-only`, or a companion `.apps.generated.ts`) before any translating."
11+
12+
Measured, the premise did not hold. Of the 77 declared keys per locale, **76
13+
were already translated** in the hand-authored `<locale>.ts` files and had been
14+
for months. Exactly one was genuinely missing —
15+
`apps.studio.navigation.nav_app_builder.label`, absent in all four locales
16+
including `en`. The 231 was a measurement artifact: this config declares
17+
SETUP_APP / STUDIO_APP / ACCOUNT_APP and SystemOverviewDashboard, but its
18+
`translations` merge baseline listed only the two GENERATED subtrees
19+
(`objects`, `metadataForms`), so coverage counted every hand-authored
20+
app/dashboard key as untranslated.
21+
22+
**Neither proposed emit is right, and the second would have caused damage.**
23+
The Setup app is a shell of empty group anchors; its ~25 menu entries are
24+
contributed at runtime by `SETUP_NAV_CONTRIBUTIONS` and by capability plugins
25+
(ADR-0029 D7). A bundle generated from a static walk of `SETUP_APP` is
26+
therefore structurally incomplete, and regenerating over the hand-authored
27+
files would have **deleted 40 live nav translations per locale**. Dropping
28+
`--objects-only` fails differently: `kind: 'full'` folds all 803 metadata-form
29+
keys into `<locale>.objects.generated.ts` and renames the export the baseline
30+
imports.
31+
32+
The split is correct as it stands and is now written down: `objects` /
33+
`metadataForms` are generated and gated by the bundle-drift check; `apps` /
34+
`dashboards` / `pages` are hand-authored and gated by the coverage ratchet.
35+
What was wrong was only that the baseline omitted the hand-authored half.
36+
37+
- Extract config's `translations` now carries the per-locale assemblers, with
38+
`objects`/`metadataForms` still pinned to the committed generated files.
39+
Safe for the emit — `--objects-only` writes `data.objects` alone, so nothing
40+
added here can reach a generated bundle, and `check:i18n` stays in sync
41+
across all nine packages.
42+
- `nav_app_builder` translated in all four locales, wording taken from the
43+
repo's own precedent for "builder" (`构建器` / `ビルダー` / `generador`).
44+
- `nav_workflows` removed from all four: its menu entry is gone from
45+
`STUDIO_APP` and nothing contributes to that app, so the translation was
46+
dead.
47+
- Coverage ratchet baselined 231 → **0**, making platform-objects the ninth
48+
package where the ratchet is a strict gate — verified to go red on a single
49+
removed translation.
50+
- A local, CLI-independent parity test walks the statically declared Studio and
51+
Account navigation plus the dashboard's widgets and asserts a translation in
52+
every locale — and the reverse, that no translation survives its nav item.
53+
Both directions verified to fail before passing.
54+
55+
An untranslated nav id is invisible in the UI — it falls back to the app's
56+
English label, so a Chinese Studio menu just shows one English entry among
57+
thirty. That is why this needed a gate rather than a one-time sweep.
58+
59+
Still out of scope: the ~25 Setup entries contributed at runtime. Bringing them
60+
under a static gate needs either an objectql dependency in this package (it
61+
depends only on spec and metadata-core) or extractor support for
62+
`navigationContributions` — a real follow-up, not something to half-do here.
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
"@objectstack/driver-sql": minor
3+
---
4+
5+
fix(driver-sql): give the bounded connection attempt an accurate error message (#3769)
6+
7+
#3781 bounded a connection attempt at 10s via `pool.createTimeoutMillis`, which
8+
stopped the 30s hang but kept knex's own wording: `Timeout acquiring a
9+
connection. The pool is probably full`. The pool is not full — the server never
10+
completed the handshake — so that message sends an operator to tune `pool.max`
11+
while the network is what is broken. This is the same defect class the boot
12+
guard in #3741 was about: an error that reads nothing like its cause.
13+
14+
`SqlDriver` now also sets the **dialect's own** connect timeout, which fails with
15+
a message that names what happened:
16+
17+
| client | key | message |
18+
|---|---|---|
19+
| `pg` / `postgres` / `postgresql` / `cockroachdb` | `connectionTimeoutMillis` | `timeout expired` |
20+
| `mysql` / `mysql2` | `connectTimeout` | `connect ETIMEDOUT` |
21+
22+
Carrying the timeout requires `connection` to be an object, so a URL string is
23+
moved into the dialect's URL slot (`connectionString` for pg, `uri` for mysql2).
24+
Verified against a black-holing listener that both forms still reach the URL's
25+
own host/port and still honour `?sslmode=require`. SQLite is untouched — opening
26+
a file has no handshake to time out.
27+
28+
**The two bounds are deliberately unequal.** They race and knex wins a tie, so
29+
equal values would let the pool timeout fire first and the accurate message would
30+
never be seen. The dialect timeout is the effective bound at **10s**; the pool
31+
timeout is a strictly looser backstop, raised from 10s to **15s**, reached only
32+
by a dialect with no connect-timeout knob or one that ignores the one we set.
33+
34+
`driver.config` keeps the shape the author passed — the rewrite applies only to
35+
what knex receives. Two existing readers depend on that: `serve.ts`'s startup
36+
banner and `createDatabase()`, which parses the URL to swap in the maintenance
37+
database. A test pins it.
38+
39+
`createDatabase()`'s own admin connection now gets the same bound; it is opened
40+
during boot against the very server we already suspect is unreachable, so it must
41+
not be the one place that still waits 30s.
42+
43+
**Migration.** None for a healthy datasource. A deployment that deliberately
44+
needs longer than 10s to establish a connection (a slow cross-region replica)
45+
sets `connection.connectionTimeoutMillis` (pg) or `connection.connectTimeout`
46+
(mysql2) explicitly, and it is left alone.
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
---
2+
"@objectstack/cli": patch
3+
---
4+
5+
fix(cli): `os validate` runs the four authoring lints `os build` runs — "validate clean, build fails" is gone (#3782)
6+
7+
`os validate` is documented, and used in CI, as the **read-only superset** of the
8+
gates `os build` runs: same checks, no artifact. It wasn't. Four authoring lints
9+
were wired into `compile.ts` only, and **two of them already fail the build**:
10+
11+
| Lint | Emits `error` | `os build` | `os validate` (before) |
12+
|---|---|---|---|
13+
| `lintAutonumberFormats` | yes |||
14+
| `lintViewRefs` (#2554) | yes |||
15+
| `lintFlowPatterns` (#1874) | not yet |||
16+
| `lintLivenessProperties` | no |||
17+
18+
So an autonumber format naming a field that doesn't exist, or a form action
19+
target naming a LIST view, passed `os validate` cleanly and then failed
20+
`os build`. Reproduced verbatim on `main` against `examples/app-todo`:
21+
`os validate` → "✓ Validation passed"; `os build` → "✗ Autonumber format
22+
validation failed". Worst for the CI setups that gate on `validate` and only
23+
discover the break at deploy time.
24+
25+
The drift was invisible for a structural reason worth naming: every *other* gate
26+
on both commands is a shared `@objectstack/lint` import, while these four are
27+
CLI-local `../utils/lint-*` modules that only `compile.ts` ever imported. Nothing
28+
made adding a gate to the build also add it to validate.
29+
30+
**The fix is two parts.** `validate.ts` now runs all four, mirroring
31+
`compile.ts`'s per-lint severity handling (`error` → exit 1, everything else →
32+
advisory, and into the `warnings` array under `--json`). And a new source-level
33+
test asserts that every `lintFoo(`/`validateFoo(` call site in `compile.ts` also
34+
appears in `validate.ts`, failing with the list of missing gates. That test is
35+
the actual fix for the class of bug — the wiring is just today's instance.
36+
37+
**What you may newly see.** `os validate` now surfaces every rule these lints
38+
carry, including the advisory ones, so existing projects can see new warnings.
39+
Only `autonumber-*` and view-reference `error` findings change the exit code —
40+
and any project they now fail was already failing `os build`.
41+
42+
`FlowLintFinding` also gains an optional `severity`, honoured by both surfaces.
43+
No rule sets it today, so flow findings stay advisory; it is the seam that lets
44+
#3760's blocking `flow-runas-unscoped` gate `os validate` and `os build`
45+
together the moment it lands, with no further wiring.

content/docs/data-modeling/drivers.mdx

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,9 @@ even earlier — that is where `MongoDBDriver` puts its
127127
pnpm add @objectstack/driver-sql pg
128128
```
129129
130-
`SqlDriver`'s constructor accepts a [Knex `Knex.Config`](https://knexjs.org/guide/#configuration-options) object verbatim.
130+
`SqlDriver`'s constructor accepts a [Knex `Knex.Config`](https://knexjs.org/guide/#configuration-options)
131+
object, passing it through unchanged except for two connect-timeout defaults
132+
described [below](#connect-timeouts).
131133
132134
```typescript
133135
import { SqlDriver } from '@objectstack/driver-sql';
@@ -152,6 +154,29 @@ Or via env var alone (driver inferred from `postgres://` scheme):
152154
export OS_DATABASE_URL=postgres://admin:secret@db.example.com:5432/myapp
153155
```
154156

157+
### Connect timeouts
158+
159+
A database endpoint that accepts the TCP connection but never completes the
160+
handshakean overloaded instance, a half-open firewall, a load balancer
161+
mid-failovermakes every query *wait* rather than fail. Left to Knex's own
162+
defaults that wait is 30 seconds per query on the request path, and with a small
163+
`pool.max` a handful of them saturate the pool. So `SqlDriver` supplies two
164+
defaults ([#3769](https://github.com/objectstack-ai/objectstack/issues/3769)):
165+
166+
| Setting | Default | Purpose |
167+
| :--- | :--- | :--- |
168+
| `connection.connectionTimeoutMillis` (pg) / `connection.connectTimeout` (mysql2) | `10_000` | The effective bound. Fails with the driver's own wording — `timeout expired` / `connect ETIMEDOUT` — which names the network. |
169+
| `pool.createTimeoutMillis` | `15_000` | Backstop, only reached by a client that has no connect-timeout option or ignores it. Deliberately looser: the two race, and Knex wins a tie, so an equal value would mask the accurate message with Knex's misleading "the pool is probably full". |
170+
171+
Set either explicitly and it is left untoucheddo that when a datasource
172+
legitimately takes longer to connect (a distant cross-region replica). SQLite
173+
gets neither: opening a file has no handshake to time out.
174+
175+
Carrying a connect timeout requires `connection` to be an object, so a URL string
176+
is moved into the client's URL slot (`connectionString` for pg, `uri` for
177+
mysql2) before reaching Knex. This affects only what Knex receivesthe config
178+
you passed is preserved as-is on the driver.
179+
155180
## MongoDB
156181

157182
Configuration properties for the MongoDB driver.

content/docs/deployment/validating-metadata.mdx

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -223,11 +223,17 @@ another package defines.
223223
| Chart bindings outside dashboards (#3583) |||
224224
| Navigation vs. granted access (ADR-0090 D6) |||
225225
| Security posture (ADR-0090 — e.g. every custom object declares `sharingModel`) |||
226+
| Autonumber `{field}` interpolation |||
227+
| View references — form targets, view-key collisions (#2554) |||
228+
| Flow authoring anti-patterns (#1874) |||
229+
| Liveness author-warnings |||
226230
| Emits `dist/objectstack.json` |||
227231

228232
So `os validate` is the fast inner-loop check (no artifact); `os build` is what
229233
you run when you need the deployable artifact. A config that passes `os validate`
230-
will not fail `os build` on schema/predicate/binding grounds. Both entry points
234+
will not fail `os build` on schema/predicate/binding grounds — a test in the CLI
235+
asserts that every gate `os build` runs is also run by `os validate`, so the two
236+
cannot drift apart again (#3782). Both entry points
231237
also check SDUI styling (ADR-0065), and `os validate` additionally runs a set of
232238
view- and page-shape checks — list-view navigation modes (ADR-0053), view
233239
container shape, and JSX/React page sources (ADR-0080/0081) — that catch UI
@@ -254,6 +260,7 @@ A clean run walks each gate and reports timing:
254260
→ Checking source-page styling (ADR-0065)...
255261
→ Checking capability references (ADR-0066)...
256262
→ Checking flow trigger wiring...
263+
→ Running authoring lints (#3782)...
257264
→ Checking security posture (ADR-0090 D7)...
258265
259266
✓ Validation passed (64ms)

0 commit comments

Comments
 (0)