Skip to content

Commit 36138c7

Browse files
baozhoutaoos-zhuangclaude
authored
feat(autonumber): date, {field} & per-scope counter reset for autonumber formats (#2043)
* feat(autonumber): date, {field} and per-scope counter reset for formats Tokenize autonumberFormat via a shared pure renderer in @objectstack/spec (parseAutonumberFormat / renderAutonumber) that both the engine fallback and the SQL driver call, so they emit byte-identical numbers (#1603 parity): - date tokens {YYYY}{YY}{MM}{DD}{YYYYMMDD} resolve the calendar day in the request's business timezone (ExecutionContext.timezone, ADR-0053; UTC fallback), threaded through new DriverOptions.timezone - {field} interpolation substitutes record values into the prefix - counter scope = rendered prefix before the sequence slot, so AD{YYYYMMDD}{0000} resets daily, {section}{island_zone}{000} numbers per group, {plan_no}{000} numbers per parent — one mechanism, no separate reset config Fixed-prefix formats (CASE-{0000}) render an empty scope and keep their single global counter. _objectstack_sequences gains a scope column (PK widened to object,tenant_id,field,scope); legacy 3-column tables migrate in place on first use, carrying existing counters to scope=''. * fix(autonumber): drop backtracking lookahead in seed scan (ReDoS) The empty-prefix legacy branch used /(\d+)(?!.*\d)/ to grab the last digit run, whose negative lookahead is a polynomial-ReDoS sink on stored values with many repeated zeros (CodeQL js/polynomial-redos, high). Replace both branches with the linear /\d+/g, preserving the last-digit-run semantics. * fix(autonumber): guard {field} interpolation footguns Add three guardrails on top of the {field}/date/per-scope autonumber work: - Empty interpolated {field} now throws (shared missingFieldValues helper) in both the SQL driver and the engine fallback, instead of silently collapsing the record into the wrong counter scope. - Build-time lint (objectstack compile): unknown / self-referencing {field} fails the build; an optional {field} warns to mark it required. - Legacy _objectstack_sequences PK-widen failure fails safe — fixed-prefix sequences keep working and a per-scope write raises an actionable error rather than an opaque DB primary-key violation at insert time. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(autonumber): hash-keyed sequence table; clarify scope & width semantics Full hardening of the remaining items from review: - #5 MySQL/length: key _objectstack_sequences by a single key_hash (SHA-256 of object,tenant_id,field,scope) instead of a 4-column natural PK. The natural PK exceeded MySQL's utf8mb4 index-length limit (a certain CREATE TABLE failure) and bounded how long a {field} scope could be. The hash PK keys every dialect uniformly and lets scope be a generous non-indexed column. Legacy 3-column and interim {scope}-column tables are migrated in place; migration fails safe (fixed-prefix keeps working, a per-scope write errors actionably). - #1 scope ambiguity: confirmed NOT fixable by separating adjacent token boundaries — when two records render the same prefix they render the same visible number, so they MUST share a counter to stay unique (a separator would mint duplicates). Documented the semantics + the remedy (delimiter literal in the format), backed by tests. The compile lint already nudges authors toward unambiguous formats. - #6 width overflow: confirmed by-design — the pad width is a MINIMUM, the counter grows past it and never wraps (mainstream autonumber semantics). Documented + regression test, no throw (throwing would break legitimate high-count sequences). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: os-zhuang <jack@objectstack.ai> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent d1129c4 commit 36138c7

13 files changed

Lines changed: 1228 additions & 45 deletions
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/objectql": minor
4+
"@objectstack/driver-sql": minor
5+
"@objectstack/cli": minor
6+
---
7+
8+
feat(autonumber): date, {field} and per-scope counter reset for autonumber formats
9+
10+
`autonumberFormat` previously only understood a single `{0000}` sequence slot —
11+
everything else was a fixed literal prefix on one global counter. Real MES/eHR
12+
record numbers need three more token classes, so the format is now tokenized by a
13+
shared pure renderer in `@objectstack/spec` (`parseAutonumberFormat` /
14+
`renderAutonumber`) that the engine fallback and the SQL driver both call, so they
15+
emit byte-identical numbers (#1603 parity):
16+
17+
- **Date tokens**`{YYYY}` `{YY}` `{MM}` `{DD}` `{YYYYMMDD}` resolve the calendar
18+
day in the request's **business timezone** (`ExecutionContext.timezone`, ADR-0053;
19+
UTC fallback), threaded through the new `DriverOptions.timezone`.
20+
- **`{field}` interpolation**`{section}{island_zone}{000}` substitutes record
21+
field values into the prefix.
22+
- **Per-scope counter reset** — the counter's scope is the rendered prefix *before*
23+
the sequence slot, so `AD{YYYYMMDD}{0000}` resets daily, `{section}{island_zone}{000}`
24+
numbers per group, and `{plan_no}{000}` numbers per parent — all from one
25+
mechanism, no separate reset config.
26+
27+
Fixed-prefix formats like `CASE-{0000}` render an empty scope and keep their single
28+
global counter, so existing sequences are unchanged. The persistent
29+
`_objectstack_sequences` table is keyed by a `key_hash` (SHA-256 of
30+
`object, tenant_id, field, scope`) — a single 64-char primary key that keys every
31+
dialect uniformly, stays within MySQL's utf8mb4 index-length limit (four raw
32+
columns would not), and lets `scope` be a generous non-indexed column. Deployments
33+
with an older table (3-column, or an interim `scope` column) are migrated in place
34+
on first use, carrying existing counters to `scope=''`.
35+
36+
Guardrails:
37+
38+
- **Empty interpolated field is a hard error, not a silent mis-number.** A
39+
`{field}` token whose value is missing at create time would render to an empty
40+
prefix and collapse the record into the wrong counter scope. Both the SQL driver
41+
and the engine fallback now refuse to generate and throw a clear error naming the
42+
empty field (shared `missingFieldValues` helper).
43+
- **Build-time lint (`@objectstack/cli compile`).** `autonumber` formats are
44+
checked against the object's fields: a `{field}` token naming a non-existent
45+
field (or the autonumber field itself) **fails the build**; a token naming an
46+
*optional* field emits an advisory warning to mark it `required: true`.
47+
- **Migration fails safe.** If a legacy table cannot be migrated to the `key_hash`
48+
shape, fixed-prefix sequences keep working via the legacy key and a per-scope
49+
write raises an actionable error instead of corrupting counters.
50+
- **Long `{field}` scopes are supported** (e.g. a long `{plan_no}`): the non-indexed
51+
`scope` column and hashed key remove the old varchar/PK length ceiling.
52+
53+
Notes on inherent semantics (documented, not bugs):
54+
55+
- The counter scope IS the rendered prefix. When two records' tokens render to the
56+
same prefix string (e.g. `{a}{b}` for `('AB','C')` and `('A','BC')`) they also
57+
render the same visible number, so they share one counter to stay unique — the
58+
remedy for genuinely-distinct groups is an unambiguous format (a delimiter
59+
literal between variable tokens).
60+
- The sequence pad width is a MINIMUM; past it the number grows (`{000}`
61+
`1000`), it never wraps — matching mainstream autonumber semantics.

packages/cli/src/commands/compile.ts

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import { lowerCallables } from '../utils/lower-callables.js';
1111
import { validateStackExpressions } from '../utils/validate-expressions.js';
1212
import { validateWidgetBindings } from '../utils/validate-widget-bindings.js';
1313
import { lintFlowPatterns } from '../utils/lint-flow-patterns.js';
14+
import { lintAutonumberFormats } from '../utils/lint-autonumber-formats.js';
1415
import { lintLivenessProperties } from '../utils/lint-liveness-properties.js';
1516
import { collectAndLintDocs } from '../utils/collect-docs.js';
1617
import { buildRuntimeBundle, cleanupOldRuntimeBundles } from '../utils/build-runtime.js';
@@ -225,6 +226,38 @@ export default class Compile extends Command {
225226
}
226227
}
227228

229+
// 3d-ter. Autonumber `{field}` interpolation lint. A format like
230+
// `{plan_no}{000}` makes the referenced field part of the counter
231+
// scope, so it must exist and be set at create time — otherwise the
232+
// runtime throws (or, unlinted, silently mis-numbers). An unknown
233+
// field is broken → fails the build; an optional field is fragile →
234+
// advisory warning. Mirrors the broken/fragile two-level guardrail.
235+
const autonumberLint = lintAutonumberFormats(result.data as Record<string, unknown>);
236+
const autonumberErrors = autonumberLint.filter((f) => f.severity === 'error');
237+
const autonumberWarnings = autonumberLint.filter((f) => f.severity === 'warning');
238+
if (autonumberErrors.length > 0) {
239+
if (flags.json) {
240+
console.log(JSON.stringify({ success: false, error: 'autonumber format validation failed', issues: autonumberErrors }));
241+
this.exit(1);
242+
}
243+
console.log('');
244+
printError(`Autonumber format validation failed (${autonumberErrors.length} issue${autonumberErrors.length > 1 ? 's' : ''})`);
245+
for (const f of autonumberErrors) {
246+
console.log(` • ${f.where}: ${f.message}`);
247+
console.log(chalk.dim(` ${f.hint}`));
248+
console.log(chalk.dim(` rule: ${f.rule}`));
249+
}
250+
this.exit(1);
251+
}
252+
if (autonumberWarnings.length > 0 && !flags.json) {
253+
console.log('');
254+
for (const f of autonumberWarnings) {
255+
printWarning(`${f.where}: ${f.message}`);
256+
console.log(chalk.dim(` ${f.hint}`));
257+
console.log(chalk.dim(` rule: ${f.rule}`));
258+
}
259+
}
260+
228261
// 3d. Package docs (ADR-0046): compile flat `src/docs/*.md` into
229262
// `docs: DocSchema[]` and lint the combined set (flatness,
230263
// namespace-prefixed names, MDX/image ban, same-package link
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { describe, it, expect } from 'vitest';
4+
import {
5+
lintAutonumberFormats,
6+
AUTONUMBER_UNKNOWN_FIELD,
7+
AUTONUMBER_OPTIONAL_FIELD,
8+
AUTONUMBER_SELF_REFERENCE,
9+
} from './lint-autonumber-formats.js';
10+
11+
describe('lintAutonumberFormats', () => {
12+
it('passes a date-only / fixed-prefix format with no {field} tokens', () => {
13+
const stack = {
14+
objects: [
15+
{ name: 'audit', fields: { audit_no: { type: 'autonumber', autonumberFormat: 'AD{YYYYMMDD}{0000}' } } },
16+
{ name: 'case', fields: { case_no: { type: 'autonumber', autonumberFormat: 'CASE-{0000}' } } },
17+
],
18+
};
19+
expect(lintAutonumberFormats(stack)).toEqual([]);
20+
});
21+
22+
it('passes when every {field} token is a required field on the object', () => {
23+
const stack = {
24+
objects: [
25+
{
26+
name: 'task',
27+
fields: {
28+
section: { type: 'text', required: true },
29+
island_zone: { type: 'text', required: true },
30+
task_no: { type: 'autonumber', autonumberFormat: '{section}{island_zone}{000}' },
31+
},
32+
},
33+
],
34+
};
35+
expect(lintAutonumberFormats(stack)).toEqual([]);
36+
});
37+
38+
it('errors when a {field} token names a non-existent field', () => {
39+
const stack = {
40+
objects: [
41+
{ name: 'task', fields: { task_no: { type: 'autonumber', autonumberFormat: '{plan_no}{000}' } } },
42+
],
43+
};
44+
const out = lintAutonumberFormats(stack);
45+
expect(out).toHaveLength(1);
46+
expect(out[0].severity).toBe('error');
47+
expect(out[0].rule).toBe(AUTONUMBER_UNKNOWN_FIELD);
48+
});
49+
50+
it('warns when a {field} token names an optional field', () => {
51+
const stack = {
52+
objects: [
53+
{
54+
name: 'task',
55+
fields: {
56+
plan_no: { type: 'text' }, // not required
57+
task_no: { type: 'autonumber', autonumberFormat: '{plan_no}{000}' },
58+
},
59+
},
60+
],
61+
};
62+
const out = lintAutonumberFormats(stack);
63+
expect(out).toHaveLength(1);
64+
expect(out[0].severity).toBe('warning');
65+
expect(out[0].rule).toBe(AUTONUMBER_OPTIONAL_FIELD);
66+
});
67+
68+
it('errors when the format interpolates the autonumber field itself', () => {
69+
const stack = {
70+
objects: [
71+
{ name: 'task', fields: { task_no: { type: 'autonumber', autonumberFormat: '{task_no}{000}' } } },
72+
],
73+
};
74+
const out = lintAutonumberFormats(stack);
75+
expect(out).toHaveLength(1);
76+
expect(out[0].severity).toBe('error');
77+
expect(out[0].rule).toBe(AUTONUMBER_SELF_REFERENCE);
78+
});
79+
80+
it('handles array-shaped fields and the `format` shorthand', () => {
81+
const stack = {
82+
objects: [
83+
{
84+
name: 'task',
85+
fields: [
86+
{ name: 'plan_no', type: 'text', required: true },
87+
{ name: 'task_no', type: 'autonumber', format: '{plan_no}{000}' },
88+
],
89+
},
90+
],
91+
};
92+
expect(lintAutonumberFormats(stack)).toEqual([]);
93+
});
94+
});
Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Build-time lint for `autonumber` field formats. An `autonumberFormat` may
5+
* interpolate other fields of the same record (`{plan_no}{000}`,
6+
* `{section}{island_zone}{000}`). That field value forms the counter SCOPE, so
7+
* if it is missing at create time the record number silently collapses into the
8+
* wrong scope — and the runtime now throws rather than emit a wrong number
9+
* (see sql-driver / engine `missingFieldValues`). This lint catches the two
10+
* ways an author (very often an AI generating templates) gets that wrong,
11+
* BEFORE it ships:
12+
*
13+
* - ERROR: `{field}` names a field that does not exist on the object — the
14+
* generation will always throw. This is broken, so it fails the build.
15+
* - WARNING: `{field}` names an OPTIONAL field — generation throws on any
16+
* record left blank. The robust shape marks the referenced field
17+
* `required: true` (mirroring ERPNext/Odoo, where a field that drives the
18+
* naming series must be mandatory). Advisory; does not fail the build.
19+
*
20+
* A self-reference (`{self}` on the autonumber field itself) is always an
21+
* ERROR — the value does not exist yet when the format renders.
22+
*/
23+
24+
import { parseAutonumberFormat, referencedFields } from '@objectstack/spec/data';
25+
26+
export interface AutonumberLintFinding {
27+
where: string;
28+
message: string;
29+
hint: string;
30+
rule: string;
31+
severity: 'error' | 'warning';
32+
}
33+
34+
type AnyRec = Record<string, unknown>;
35+
36+
export const AUTONUMBER_UNKNOWN_FIELD = 'autonumber-references-unknown-field';
37+
export const AUTONUMBER_OPTIONAL_FIELD = 'autonumber-references-optional-field';
38+
export const AUTONUMBER_SELF_REFERENCE = 'autonumber-references-self';
39+
40+
function asArray(v: unknown): AnyRec[] {
41+
if (Array.isArray(v)) return v as AnyRec[];
42+
if (v && typeof v === 'object') {
43+
return Object.entries(v as AnyRec).map(([name, def]) => ({ name, ...(def as AnyRec) }));
44+
}
45+
return [];
46+
}
47+
48+
/**
49+
* Lint every `autonumber` field's format for unresolvable / fragile `{field}`
50+
* interpolation. Returns a (possibly empty) list of findings; never throws.
51+
*/
52+
export function lintAutonumberFormats(stack: AnyRec): AutonumberLintFinding[] {
53+
const findings: AutonumberLintFinding[] = [];
54+
for (const obj of asArray(stack.objects)) {
55+
const objectName = typeof obj.name === 'string' ? obj.name : '(unnamed object)';
56+
const fields = asArray(obj.fields);
57+
// name → required?, for schema-aware reference checks.
58+
const fieldMeta = new Map<string, { required: boolean }>();
59+
for (const f of fields) {
60+
if (typeof f.name === 'string') fieldMeta.set(f.name, { required: f.required === true });
61+
}
62+
63+
for (const f of fields) {
64+
if (f.type !== 'autonumber') continue;
65+
const name = typeof f.name === 'string' ? f.name : '(unnamed field)';
66+
const fmt = typeof f.autonumberFormat === 'string'
67+
? f.autonumberFormat
68+
: (typeof f.format === 'string' ? f.format : '');
69+
if (!fmt) continue;
70+
const refs = referencedFields(parseAutonumberFormat(fmt));
71+
const where = `object '${objectName}' · field '${name}' (autonumber "${fmt}")`;
72+
for (const ref of refs) {
73+
if (ref === name) {
74+
findings.push({
75+
where,
76+
message: `format interpolates \`{${ref}}\` — its own value, which does not exist yet when the number is generated.`,
77+
hint: `Reference a DIFFERENT field that is set before create (e.g. \`{plan_no}{000}\`), or drop the token.`,
78+
rule: AUTONUMBER_SELF_REFERENCE,
79+
severity: 'error',
80+
});
81+
continue;
82+
}
83+
const meta = fieldMeta.get(ref);
84+
if (!meta) {
85+
findings.push({
86+
where,
87+
message: `format interpolates \`{${ref}}\`, but object '${objectName}' has no field named '${ref}' — generation will always throw.`,
88+
hint: `Reference an existing field, or remove the \`{${ref}}\` token from the format.`,
89+
rule: AUTONUMBER_UNKNOWN_FIELD,
90+
severity: 'error',
91+
});
92+
} else if (!meta.required) {
93+
findings.push({
94+
where,
95+
message: `format interpolates \`{${ref}}\`, but '${ref}' is optional — any record left blank fails autonumber generation at create time.`,
96+
hint: `Mark '${ref}' as \`required: true\` so it is always set before the record number is rendered.`,
97+
rule: AUTONUMBER_OPTIONAL_FIELD,
98+
severity: 'warning',
99+
});
100+
}
101+
}
102+
}
103+
}
104+
return findings;
105+
}

packages/objectql/src/engine-autonumber-defer.test.ts

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,4 +112,61 @@ describe('ObjectQL autonumber ownership (#1603)', () => {
112112
expect(driver.created[0].doc_no).toBe('D-0001');
113113
expect(result.doc_no).toBe('D-0001');
114114
});
115+
116+
// The fallback path renders the SAME format tokens as the SQL driver
117+
// (shared @objectstack/spec renderer), so {field}/{date} grouping must match.
118+
it('fallback renders {field} tokens and counts independently per scope', async () => {
119+
const TASK_SCHEMA = {
120+
name: 'task',
121+
fields: {
122+
zone: { type: 'text' },
123+
task_no: { type: 'autonumber', format: '{zone}{000}' },
124+
},
125+
};
126+
vi.mocked(SchemaRegistry.getObject).mockReturnValue(TASK_SCHEMA as any);
127+
const driver = makeDriver(false);
128+
engine.registerDriver(driver, true);
129+
await engine.init();
130+
131+
const a1 = await engine.insert('task', { zone: 'A' });
132+
const b1 = await engine.insert('task', { zone: 'B' });
133+
const a2 = await engine.insert('task', { zone: 'A' });
134+
135+
expect(a1.task_no).toBe('A001');
136+
expect(b1.task_no).toBe('B001'); // a different scope restarts at 001
137+
expect(a2.task_no).toBe('A002');
138+
});
139+
140+
it('fallback renders {YYYYMMDD} date tokens in the business timezone', async () => {
141+
const AUDIT_SCHEMA = {
142+
name: 'audit',
143+
fields: { audit_no: { type: 'autonumber', format: 'AD{YYYYMMDD}{0000}' } },
144+
};
145+
vi.mocked(SchemaRegistry.getObject).mockReturnValue(AUDIT_SCHEMA as any);
146+
const driver = makeDriver(false);
147+
engine.registerDriver(driver, true);
148+
await engine.init();
149+
150+
const r = await engine.insert('audit', {}, { timezone: 'UTC' } as any);
151+
// Today's UTC day + a fresh per-day counter.
152+
expect(r.audit_no).toMatch(/^AD\d{8}0001$/);
153+
});
154+
155+
it('fallback refuses to generate when an interpolated {field} is empty', async () => {
156+
const TASK_SCHEMA = {
157+
name: 'task',
158+
fields: {
159+
zone: { type: 'text' },
160+
task_no: { type: 'autonumber', format: '{zone}{000}' },
161+
},
162+
};
163+
vi.mocked(SchemaRegistry.getObject).mockReturnValue(TASK_SCHEMA as any);
164+
const driver = makeDriver(false);
165+
engine.registerDriver(driver, true);
166+
await engine.init();
167+
168+
// zone left blank → the prefix collapses to '' and would mis-scope the
169+
// counter, so generation must throw rather than emit a wrong number.
170+
await expect(engine.insert('task', {})).rejects.toThrow(/zone/);
171+
});
115172
});

0 commit comments

Comments
 (0)