Skip to content

Commit 29ff3c2

Browse files
os-zhuangclaude
andauthored
feat(lint): warn on replay-unsafe mode:'insert' seeds + document composite externalId (#3460)
Follow-up to #3442 (which closed #3434). - lint: add validateSeedReplaySafety (@objectstack/lint, a pure (stack) => Finding[] rule per ADR-0019) and wire it into os validate / os lint. It flags every data[] seed declared mode:'insert' — the one non-idempotent mode, which duplicates its table on every replay boot — and points the author at ignore/upsert plus an externalId (single field, or a composite list for a join table). Catches the #3434 footgun at authoring time. - docs: correct the seed insert-mode row in seed-data.mdx, add an insert replay-safety warning, and add a "Composite externalId — Join / Junction Tables" section plus best-practice and API-reference updates. Verified on a real better-sqlite3 file DB with the real showcase ProjectMembership object: mode:'insert' reproduces 3 -> 6 -> 9 across reboots, while mode:'ignore' + externalId:['team','project'] stays 3 -> 3 -> 3. Refs #3434, #3442 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JqZsDkKaCkpQGpJdEMEWDF
1 parent 84e7be9 commit 29ff3c2

6 files changed

Lines changed: 229 additions & 3 deletions

File tree

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
"@objectstack/lint": patch
3+
"@objectstack/cli": patch
4+
---
5+
6+
feat(lint): warn on replay-unsafe `mode: 'insert'` seed datasets (#3434 follow-up)
7+
8+
Seeds are replayed — they re-load on every dev-server boot and every package
9+
re-publish, not applied once — so `mode: 'insert'` (the loader's one mode with
10+
no existing-row check) duplicates its table on every restart. That footgun
11+
shipped undetected until #3434 (showcase memberships grew 3 → 6 → 9).
12+
13+
Adds `validateSeedReplaySafety` to `@objectstack/lint` (a pure `(stack) => Finding[]`
14+
rule, ADR-0019) and wires it into `os validate` / `os lint`. Every `data[]` seed
15+
declared with `mode: 'insert'` now gets an advisory warning that points at the
16+
idempotent modes (`ignore` / `upsert`) and the `externalId` to match on — a
17+
single natural-key field, or a COMPOSITE list of fields for a join / junction
18+
table with no single key (`['team', 'project']`, the support #3434 added). It
19+
catches the mistake at authoring time instead of on the second boot.

content/docs/data-modeling/seed-data.mdx

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ record (matched by `externalId`).
6565
| Mode | Behavior | Use Case |
6666
|:-----|:---------|:---------|
6767
| `upsert` | Create if new, update if found | Default — idempotent for most data |
68-
| `insert` | Create only, throw on duplicate | Append-only tables, audit logs |
68+
| `insert` | Insert **every** record unconditionally — no existing-row check, so it **duplicates the table on each replay boot** ([#3434](https://github.com/objectstack-ai/objectstack/issues/3434)). Rarely what you want | Append-only rows loaded exactly once (never re-run) |
6969
| `update` | Update only, skip if not found | Migration patches on existing rows |
7070
| `ignore` | Create if new, silently skip duplicates | Bootstrap data that must not overwrite user edits |
7171
| `replace` | Insert without checking for an existing match — the seed loader does **not** delete anything itself; the target table must already be empty (or cleared by the caller) | Cache / lookup tables rebuilt from an already-truncated table |
@@ -115,6 +115,15 @@ defineSeed(ExchangeRateCache, {
115115
});
116116
```
117117

118+
### `insert` — Not Idempotent (Avoid for Replayed Seeds)
119+
120+
Seeds are **replayed** — they re-load on every dev-server boot and every package
121+
re-publish, not applied once. `insert` writes every record with **no existing-row
122+
check**, so a replayed `insert` dataset duplicates its table on each restart
123+
(a 3-row fixture becomes 6, then 9 — [#3434](https://github.com/objectstack-ai/objectstack/issues/3434)).
124+
It is almost never the right mode: reach for `upsert` (or `ignore`) with a stable
125+
`externalId` instead. `os validate` warns on any `mode: 'insert'` seed.
126+
118127
---
119128

120129
## Environment Scoping
@@ -210,6 +219,31 @@ const contactsSeed = defineSeed(Contact, {
210219
export const SeedData = [accountsSeed, contactsSeed];
211220
```
212221

222+
### Composite `externalId` — Join / Junction Tables
223+
224+
A join (junction) table linking two objects many-to-many has **no single-field
225+
natural key** — the *pair* of foreign keys is what makes a row unique. Pass a
226+
**list** of field names as the `externalId` so the seed runner can match existing
227+
rows on the composite key and stay idempotent across replays. The foreign keys are
228+
compared by their *resolved* record ids, which are stable between boots.
229+
230+
```typescript
231+
// A team can staff many projects; a project can have many teams.
232+
// `showcase_project_membership` is the join row — unique on (team, project).
233+
const membershipsSeed = defineSeed(ProjectMembership, {
234+
mode: 'ignore', // skip pairs that already exist
235+
externalId: ['team', 'project'], // composite natural key — both FKs
236+
records: [
237+
{ team: 'Experience', project: 'Website Relaunch', engagement: 'owner' },
238+
{ team: 'Platform', project: 'Data Platform', engagement: 'owner' },
239+
{ team: 'Platform', project: 'Website Relaunch', engagement: 'contributor' },
240+
],
241+
});
242+
```
243+
244+
Without a composite key such a table can only fall back to `mode: 'insert'`, which
245+
duplicates on every replay boot ([#3434](https://github.com/objectstack-ai/objectstack/issues/3434)).
246+
213247
---
214248

215249
## Dynamic Values (CEL)
@@ -353,6 +387,7 @@ databases.
353387
| User records | `'email'` |
354388
| Generic named records | `'name'` (default) |
355389
| Externally sourced data | `'external_id'` |
390+
| Join / junction tables (no single natural key) | a composite list, e.g. `['team', 'project']` |
356391

357392
### Scope demo data with `env`
358393

@@ -391,7 +426,7 @@ function defineSeed<
391426
>(
392427
objectDef: TObj,
393428
config: {
394-
externalId?: string; // default: 'name'
429+
externalId?: string | string[]; // single field, or a composite list (join tables); default: 'name'
395430
mode?: 'insert' | 'update' | 'upsert' | 'replace' | 'ignore'; // default: 'upsert'
396431
env?: Array<'prod' | 'dev' | 'test'>; // default: ['prod','dev','test']
397432
records: Array<Partial<Record<keyof TObj['fields'], unknown>>>;

packages/cli/src/commands/lint.ts

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import { loadConfig, BUNDLE_REQUIRE_EXTERNALS } from '../utils/config.js';
99
import { computeI18nCoverage, type CoverageIssue } from '../utils/i18n-coverage.js';
1010
import { lintDataModel } from '../lint/data-model-rules.js';
1111
import { validateWidgetBindings } from '@objectstack/lint';
12-
import { validateRecordTitle, validateSemanticRoles, validateCapabilityReferences, validateSecurityPosture, validateApprovalApprovers } from '@objectstack/lint';
12+
import { validateRecordTitle, validateSemanticRoles, validateCapabilityReferences, validateSecurityPosture, validateApprovalApprovers, validateSeedReplaySafety } from '@objectstack/lint';
1313
import { collectAndLintDocs } from '../utils/collect-docs.js';
1414
import { scoreMetadata } from '../lint/score.js';
1515
import { runMetadataEval } from '../lint/metadata-eval.js';
@@ -440,6 +440,21 @@ export function lintConfig(config: any): LintIssue[] {
440440
});
441441
}
442442

443+
// ── Seed replay safety (framework#3434) ──
444+
// Seeds are replayed on every boot / re-publish, so a `mode: 'insert'` dataset
445+
// duplicates its table on every restart (the loader's insert path has no
446+
// existing-row check). Advisory: the fix-it points at `ignore`/`upsert` + an
447+
// `externalId` (single field, or a composite list for a join table).
448+
for (const t of validateSeedReplaySafety(config)) {
449+
issues.push({
450+
severity: t.severity,
451+
rule: t.rule,
452+
message: `${t.where}: ${t.message}`,
453+
path: t.path,
454+
fix: t.hint,
455+
});
456+
}
457+
443458
return issues;
444459
}
445460

packages/lint/src/index.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,12 @@ export {
110110
} from './validate-approval-approvers.js';
111111
export type { ApprovalApproverFinding, ApprovalApproverSeverity } from './validate-approval-approvers.js';
112112

113+
export {
114+
validateSeedReplaySafety,
115+
SEED_INSERT_MODE_DUPLICATES_ON_REPLAY,
116+
} from './validate-seed-replay-safety.js';
117+
export type { SeedReplaySafetyFinding, SeedReplaySafetySeverity } from './validate-seed-replay-safety.js';
118+
113119
export {
114120
validateSecurityPosture,
115121
SECURITY_OWD_UNSET,
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
import { describe, it, expect } from 'vitest';
2+
import {
3+
validateSeedReplaySafety,
4+
SEED_INSERT_MODE_DUPLICATES_ON_REPLAY,
5+
} from './validate-seed-replay-safety.js';
6+
7+
describe('validateSeedReplaySafety (framework#3434 replay-unsafe seed gate)', () => {
8+
it('passes a stack with no seed data', () => {
9+
expect(validateSeedReplaySafety({})).toHaveLength(0);
10+
expect(validateSeedReplaySafety({ data: [] })).toHaveLength(0);
11+
});
12+
13+
it('passes idempotent modes (ignore / upsert / update / replace) and the default (unset)', () => {
14+
const findings = validateSeedReplaySafety({
15+
data: [
16+
{ object: 'a', mode: 'ignore', externalId: ['team', 'project'], records: [] },
17+
{ object: 'b', mode: 'upsert', externalId: 'code', records: [] },
18+
{ object: 'c', mode: 'update', records: [] },
19+
{ object: 'd', mode: 'replace', records: [] },
20+
{ object: 'e', records: [] }, // mode unset → defaults to upsert, safe
21+
],
22+
});
23+
expect(findings).toHaveLength(0);
24+
});
25+
26+
it("flags a mode: 'insert' seed with location + an actionable fix hint", () => {
27+
const findings = validateSeedReplaySafety({
28+
data: [
29+
{ object: 'showcase_project_membership', mode: 'insert', records: [{ team: 'x', project: 'y' }] },
30+
],
31+
});
32+
expect(findings).toHaveLength(1);
33+
expect(findings[0]).toMatchObject({
34+
severity: 'warning',
35+
rule: SEED_INSERT_MODE_DUPLICATES_ON_REPLAY,
36+
path: 'data[0].mode',
37+
});
38+
expect(findings[0].where).toContain('showcase_project_membership');
39+
expect(findings[0].message).toContain('replay');
40+
// Hint steers to the idempotent modes and the composite externalId the fix added.
41+
expect(findings[0].hint).toContain('ignore');
42+
expect(findings[0].hint).toContain('upsert');
43+
expect(findings[0].hint).toContain('externalId');
44+
expect(findings[0].hint).toContain("['team', 'project']");
45+
});
46+
47+
it('flags each insert seed and reports its index in the path; leaves safe seeds alone', () => {
48+
const findings = validateSeedReplaySafety({
49+
data: [
50+
{ object: 'safe', mode: 'upsert', records: [] },
51+
{ object: 'bad_one', mode: 'insert', records: [] },
52+
{ object: 'also_safe', mode: 'ignore', records: [] },
53+
{ object: 'bad_two', mode: 'insert', records: [] },
54+
],
55+
});
56+
expect(findings).toHaveLength(2);
57+
expect(findings.map((f) => f.path)).toEqual(['data[1].mode', 'data[3].mode']);
58+
expect(findings.map((f) => f.where)).toEqual(['seed "bad_one"', 'seed "bad_two"']);
59+
});
60+
61+
it('falls back to a data[i] location when a seed has no object name', () => {
62+
const findings = validateSeedReplaySafety({ data: [{ mode: 'insert', records: [] }] });
63+
expect(findings).toHaveLength(1);
64+
expect(findings[0].where).toBe('data[0]');
65+
});
66+
67+
it('ignores non-object entries defensively', () => {
68+
const findings = validateSeedReplaySafety({ data: [null, 'insert', 42, { object: 'z', mode: 'insert', records: [] }] });
69+
expect(findings).toHaveLength(1);
70+
expect(findings[0].where).toContain('z');
71+
});
72+
});
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// Build-time guardrail for replay-unsafe seed datasets (framework#3434).
4+
//
5+
// A pure `(stack) => Finding[]` rule (ADR-0019), run from `os validate` and
6+
// reusable by AI authoring. Seeds are REPLAYED — they re-load on every
7+
// dev-server boot and every package re-publish, not applied once — so a
8+
// dataset's mode has to be idempotent. `mode: 'insert'` is the one mode that
9+
// is not: the loader's `insert` path writes every record unconditionally, with
10+
// no existing-row check, so the table grows by the dataset's size on every
11+
// restart (the showcase `showcase_project_membership` fixture went 3 → 6 → 9).
12+
//
13+
// This is the authoring-time nudge that would have caught #3434 before boot:
14+
// flag `insert`, and point at the idempotent modes (`ignore` / `upsert`) plus
15+
// the `externalId` — single field, or a COMPOSITE list of fields for a join /
16+
// junction table that has no single natural key (`['team', 'project']`), the
17+
// support for which #3434 added.
18+
//
19+
// Advisory (warning): an `insert` seed is not a schema error — it loads and
20+
// "works" on a fresh DB; the defect only shows on the second boot. So it earns
21+
// a located fix-it, not a hard `os compile` gate.
22+
23+
export type SeedReplaySafetySeverity = 'error' | 'warning';
24+
25+
export interface SeedReplaySafetyFinding {
26+
severity: SeedReplaySafetySeverity;
27+
rule: string;
28+
/** Human-readable location, e.g. `seed "showcase_project_membership"`. */
29+
where: string;
30+
/** Config path, e.g. `data[12].mode`. */
31+
path: string;
32+
message: string;
33+
hint: string;
34+
}
35+
36+
// Rule id (registry entry).
37+
export const SEED_INSERT_MODE_DUPLICATES_ON_REPLAY = 'seed-insert-mode-duplicates-on-replay';
38+
39+
type AnyRec = Record<string, unknown>;
40+
41+
/**
42+
* Flag every seed dataset declared with `mode: 'insert'` — the one non-idempotent
43+
* mode, which duplicates its rows on every replay boot (framework#3434). Returns
44+
* the findings (empty = clean). The caller decides how to surface them / whether
45+
* to fail the build; the CLI folds them in as advisory warnings.
46+
*
47+
* Reads `stack.data` (the `SeedSchema[]` fixtures). Safe on any shape — a stack
48+
* with no `data` array yields no findings.
49+
*/
50+
export function validateSeedReplaySafety(stack: AnyRec): SeedReplaySafetyFinding[] {
51+
const out: SeedReplaySafetyFinding[] = [];
52+
const seeds = Array.isArray(stack.data) ? (stack.data as AnyRec[]) : [];
53+
54+
seeds.forEach((seed, i) => {
55+
if (!seed || typeof seed !== 'object') return;
56+
if (seed.mode !== 'insert') return;
57+
58+
const object = typeof seed.object === 'string' ? seed.object : undefined;
59+
const where = object ? `seed "${object}"` : `data[${i}]`;
60+
61+
out.push({
62+
severity: 'warning',
63+
rule: SEED_INSERT_MODE_DUPLICATES_ON_REPLAY,
64+
where,
65+
path: `data[${i}].mode`,
66+
message:
67+
"`mode: 'insert'` re-inserts every record on each replay boot (dev-server restart, " +
68+
'package re-publish) with no existing-row check, so the dataset duplicates the table ' +
69+
'on every restart — seeds are replayed, not applied once.',
70+
hint:
71+
"Use `mode: 'ignore'` (skip rows that already exist) or `'upsert'` (create-or-update), " +
72+
"and declare an `externalId` to match on: a single natural-key field (e.g. `externalId: 'code'`), " +
73+
'or a COMPOSITE list of fields for a join / junction table with no single natural key ' +
74+
"(e.g. `externalId: ['team', 'project']`).",
75+
});
76+
});
77+
78+
return out;
79+
}

0 commit comments

Comments
 (0)