Skip to content

Commit 41053dc

Browse files
committed
feat(automation,formula): warn on undeclared flow-node config keys (#4045)
`FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or invented config key was accepted in total silence. `visibleIf` instead of `visibleWhen` registered cleanly, was never read, and the only symptom was a feature that quietly did not happen — the diagnostic vacuum that made #3528 take three passes and two wrong diagnoses. `registerFlow` now walks each node's `config` against its descriptor's `configSchema` and warns on anything undeclared, located to the exact path with the declared set listed. The walk descends where the schema declares structure and STOPS at free-form keyValue maps, whose keys are author data (`filter: { status: 'stale' }`). Descending is load-bearing, not thoroughness: `visibleWhen` is a property of `screen`'s `fields[].items`, NOT of config's top level, so a top-level-only comparison would miss the exact typo class this exists to catch. A test pins `config.fields[0].visibleIf`. Warn, never reject. An undeclared key is one of three things and this seam cannot yet tell them apart: an author typo (reject-worthy), a key the executor genuinely reads that its hand-written configSchema never declared (`notify.source` was exactly this until #4050 — rejecting those breaks working apps), or dead config. Only 4 of the 13 schema-carrying builtins have been audited for the second population, so hard-failing would gamble on the other nine. This warning is what measures the distribution; tightening is a later per-key decision, and belongs with a tombstone carrying the prescription (the `UNKNOWN_KEY_GUIDANCE` pattern). Soft-fail matches `validateNodeTypes` immediately above — same function, same channel, same reasoning. The published `configSchema` is unchanged, so no consumer sees a different shape. `@objectstack/formula` exports `nearestName`, the edit-distance helper already behind unknown-field and unknown-role suggestions, so these diagnostics share one threshold rather than each inventing its own. It is deliberately a bonus, not the mechanism: `visibleIf` → `visibleWhen` is distance 4 against a threshold of 3, so the declared set is printed ALWAYS. Loosening the shared threshold would mean confidently wrong suggestions over the hundreds of candidates the field-ref diagnostic faces; a config object declares at most a dozen keys, so listing them is cheap and complete. Both branches are tested. Measured against the repo's own apps before shipping: 30 flows across app-crm and app-showcase produce exactly ONE warning — no flood — and that one is a real finding, fixed here. `showcase_inquiry_purge`'s `get_record` carried `mode: 'records'`, which no executor reads (`if (limit && limit > 1)` selects the list read), with a comment crediting `mode` for what `limit` actually does. Left as is, our own showcase would warn on every dev boot and keep advertising a `mode` idiom an AI author would copy. Test plan: formula 265 passed, service-automation 448 passed (8 new), app-showcase `validate` + `typecheck` clean, ESLint clean on all changed files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QoA8AV99Ss1RRkLLAYmDzq
1 parent 01e124d commit 41053dc

6 files changed

Lines changed: 347 additions & 6 deletions

File tree

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
'@objectstack/service-automation': minor
3+
'@objectstack/formula': minor
4+
---
5+
6+
Warn on flow-node `config` keys the node type does not declare (#4045).
7+
8+
`FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or invented
9+
config key was accepted in total silence: `visibleIf` instead of `visibleWhen`
10+
registered cleanly, was never read, and the only symptom was a feature that quietly
11+
did not happen. That diagnostic vacuum is what made #3528 take three passes and two
12+
wrong diagnoses to resolve.
13+
14+
`registerFlow` now compares each node's `config` against its descriptor's
15+
`configSchema` and warns on anything undeclared, located and with the declared set
16+
listed:
17+
18+
```
19+
[flow 'lead_conversion'] node 'screen_1' (screen): unknown config key `visibleIf`
20+
at config.fields[0].visibleIf — It is not declared by this node type's
21+
configSchema, so nothing reads it. Declared here: name, label, type, required,
22+
visibleWhen.
23+
```
24+
25+
The walk descends where the schema declares structure and **stops at free-form
26+
keyValue maps**, whose keys are author data (`filter: { status: 'stale' }`).
27+
Descending matters: the #3528 typo class lives *inside* the `screen` field
28+
repeater, so a top-level-only comparison would miss the exact mistake this exists
29+
to catch.
30+
31+
**Warn, never reject.** An undeclared key is an author typo, a key the executor
32+
genuinely reads that its hand-written `configSchema` never declared (`notify.source`
33+
was exactly this), or dead config. Only 4 of the 13 schema-carrying builtins have
34+
been audited for the second population, so hard-failing would gamble on the other
35+
nine. Tightening to an error is a later, per-key decision once this warning has
36+
measured the real distribution. Nothing about the published `configSchema` changes,
37+
so no consumer sees a different shape.
38+
39+
`@objectstack/formula` now exports `nearestName`, the edit-distance helper already
40+
used for unknown-field and unknown-role suggestions, so "did you mean?"
41+
diagnostics share one threshold. It is deliberately a bonus rather than the
42+
mechanism — `visibleIf``visibleWhen` is distance 4 against a threshold of 3, so
43+
the declared set is always listed instead of only as a fallback.
44+
45+
Also fixes the first real finding from the new check: `showcase_inquiry_purge`'s
46+
`get_record` node carried `mode: 'records'`, which no executor reads, with a comment
47+
crediting it for behaviour that `limit > 1` actually produces.

examples/app-showcase/src/automation/flows/index.ts

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1357,13 +1357,16 @@ export const InquiryPurgeFlow = defineFlow({
13571357
id: 'purge_check',
13581358
type: 'get_record',
13591359
label: 'Find closed inquiries',
1360-
// records mode + outputVariable: the result lands in the variable
1361-
// context, where the out-edge CEL below reads it (the engine routes on
1362-
// EDGE conditions — same contract as BudgetApprovalFlow's gate).
1360+
// `limit > 1` is what makes this a LIST read (`find`, not `findOne`) — the
1361+
// executor has no `mode` key, so a `mode: 'records'` used to sit here doing
1362+
// nothing while the comment credited it for the behaviour `limit` actually
1363+
// produced (found by the #4045 unknown-config-key check). `outputVariable`
1364+
// then lands the array in the variable context, where the out-edge CEL below
1365+
// reads it — the engine routes on EDGE conditions, same contract as
1366+
// BudgetApprovalFlow's gate.
13631367
config: {
13641368
objectName: 'showcase_inquiry',
13651369
filter: { status: 'closed' },
1366-
mode: 'records',
13671370
limit: 200,
13681371
outputVariable: 'closedInquiries',
13691372
},

packages/formula/src/index.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ export type { CelFilterCompileResult, CelFilterCompileOptions, CelFilterFailReas
2828
export { matchesFilterCondition } from './matches-filter';
2929
// ADR-0032 — shared validator + introspection (one validator for build,
3030
// registration, and the agent-callable validate_expression tool).
31-
export { validateExpression, introspectScope, expectedDialect, inferExpressionType, CEL_STDLIB_FUNCTIONS } from './validate';
31+
export { validateExpression, introspectScope, expectedDialect, inferExpressionType, nearestName, CEL_STDLIB_FUNCTIONS } from './validate';
3232
export type { FieldRole, ExprInput, ExprSchemaHint, ExprValidationError, ExprValidationResult, InferredValueType } from './validate';
3333
export type { SeedValue, SeedPrimitive } from './seed-eval';
3434
export type { DialectEngine, EvalContext, EvalResult, EvalError } from './types';

packages/formula/src/validate.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,20 @@ function checkFieldExistence(source: string, schema: ExprSchemaHint | undefined,
216216
}
217217

218218
/** Cheap edit-distance suggestion for typo'd field names. */
219+
/**
220+
* The closest candidate to `name`, or `undefined` when nothing is close enough
221+
* to be worth suggesting.
222+
*
223+
* The public face of the same edit-distance heuristic this module already uses
224+
* for unknown field refs and unknown roles, exported so other "did you mean?"
225+
* diagnostics reuse one threshold instead of each inventing their own — an
226+
* author (increasingly an agent) should not get a suggestion here and silence
227+
* there for the same class of typo.
228+
*/
229+
export function nearestName(name: string, candidates: readonly string[]): string | undefined {
230+
return nearest(name, candidates);
231+
}
232+
219233
function nearest(name: string, candidates: readonly string[]): string | undefined {
220234
let best: string | undefined;
221235
let bestD = Infinity;
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Unknown flow-node `config` keys are reported at registration (#4045).
5+
*
6+
* `FlowNodeSchema.config` is `z.record(z.unknown())`, so before this a misspelled
7+
* or invented config key was accepted in **total silence**: `visibleIf` instead of
8+
* `visibleWhen` registered cleanly, was never read, and the only symptom was a
9+
* feature that quietly did not happen. That is the diagnostic vacuum that made
10+
* #3528 take three passes and two wrong diagnoses.
11+
*
12+
* Warn, never reject — see `validateNodeConfigKeys` for why. These tests pin both
13+
* halves of that: the warning fires with an actionable suggestion, AND
14+
* registration still succeeds.
15+
*/
16+
17+
import { describe, it, expect } from 'vitest';
18+
import { AutomationEngine } from '../engine.js';
19+
import { installBuiltinNodes } from './index.js';
20+
21+
function recordingLogger() {
22+
const warnings: string[] = [];
23+
const logger: any = {
24+
info() {}, error() {}, debug() {},
25+
warn(msg: string) { warnings.push(msg); },
26+
child() { return logger; },
27+
};
28+
return { logger, warnings };
29+
}
30+
31+
function engineWith(logger: any) {
32+
const engine = new AutomationEngine(logger);
33+
installBuiltinNodes(engine, { logger, getService() { throw new Error('none'); } } as any);
34+
return engine;
35+
}
36+
37+
/** A one-node flow carrying `config` verbatim on a node of `type`. */
38+
function flowWith(type: string, config: Record<string, unknown>) {
39+
return {
40+
name: 'f', label: 'F', type: 'screen', status: 'active', version: 1,
41+
nodes: [
42+
{ id: 'start', type: 'start', label: 'S', config: {} },
43+
{ id: 'n1', type, label: 'N', config },
44+
{ id: 'end', type: 'end', label: 'E' },
45+
],
46+
edges: [
47+
{ id: 'e1', source: 'start', target: 'n1', type: 'default' },
48+
{ id: 'e2', source: 'n1', target: 'end', type: 'default' },
49+
],
50+
};
51+
}
52+
53+
describe('unknown node config keys (#4045)', () => {
54+
it('flags a misspelled key with a did-you-mean suggestion', () => {
55+
const { logger, warnings } = recordingLogger();
56+
const engine = engineWith(logger);
57+
58+
// The typo lives INSIDE the field repeater — where the real #3528 one did.
59+
engine.registerFlow('f', flowWith('screen', {
60+
fields: [{ name: 'opportunityName', required: true, visibleIf: 'createOpportunity == true' }],
61+
}));
62+
63+
const hit = warnings.find((w) => w.includes('visibleIf'));
64+
expect(hit, 'the typo should be reported').toBeDefined();
65+
expect(hit).toContain("node 'n1'");
66+
expect(hit).toContain('(screen)');
67+
// Located to the exact element, so the author knows WHICH field.
68+
expect(hit).toContain('config.fields[0].visibleIf');
69+
// The load-bearing half for an agent author: the correct key is named. Note
70+
// it comes from the DECLARED SET, not the suggestion — `visibleIf` →
71+
// `visibleWhen` is edit-distance 4 against `nearestName`'s threshold of 3, so
72+
// this exact typo gets no did-you-mean. Printing the declared set is what
73+
// makes the diagnostic actionable regardless.
74+
expect(hit).toContain('Declared here: name, label, type, required, visibleWhen.');
75+
});
76+
77+
it('adds a did-you-mean when the typo IS within edit distance', () => {
78+
const { logger, warnings } = recordingLogger();
79+
const engine = engineWith(logger);
80+
81+
engine.registerFlow('f', flowWith('screen', { titl: 'Details' }));
82+
83+
const hit = warnings.find((w) => w.includes('titl'));
84+
expect(hit).toContain('did you mean `title`?');
85+
// …and still lists the declared set alongside it.
86+
expect(hit).toContain('Declared here:');
87+
});
88+
89+
it('still registers the flow — warn, never reject', () => {
90+
const { logger } = recordingLogger();
91+
const engine = engineWith(logger);
92+
expect(() => engine.registerFlow('f', flowWith('screen', { visibleIf: 'a == true' }))).not.toThrow();
93+
// And it is really registered, not swallowed.
94+
expect(engine.getFlow('f')).toBeDefined();
95+
});
96+
97+
it('lists the declared keys when nothing is close enough to suggest', () => {
98+
const { logger, warnings } = recordingLogger();
99+
const engine = engineWith(logger);
100+
101+
engine.registerFlow('f', flowWith('screen', { totallyMadeUp: 42 }));
102+
103+
const hit = warnings.find((w) => w.includes('totallyMadeUp'));
104+
expect(hit).toBeDefined();
105+
expect(hit).not.toContain('did you mean');
106+
// The declared set is always present, so there is always somewhere to go.
107+
expect(hit).toContain('Declared here:');
108+
expect(hit).toContain('fields');
109+
});
110+
111+
it('reports every undeclared key, not just the first', () => {
112+
const { logger, warnings } = recordingLogger();
113+
const engine = engineWith(logger);
114+
115+
engine.registerFlow('f', flowWith('screen', { hideWhen: 'x', submitLabel: 'Go' }));
116+
117+
expect(warnings.some((w) => w.includes('hideWhen'))).toBe(true);
118+
expect(warnings.some((w) => w.includes('submitLabel'))).toBe(true);
119+
});
120+
121+
it('stays quiet on a fully declared config', () => {
122+
const { logger, warnings } = recordingLogger();
123+
const engine = engineWith(logger);
124+
125+
engine.registerFlow('f', flowWith('screen', {
126+
title: 'Details',
127+
fields: [{ name: 'a', type: 'boolean', required: true, visibleWhen: 'b == true' }],
128+
waitForInput: true,
129+
defaults: { x: 1 },
130+
}));
131+
132+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
133+
});
134+
135+
it('stays quiet for a node type that publishes no configSchema', () => {
136+
// `decision` / `script` are deliberately schemaless (config-schemas.test.ts):
137+
// nothing is declared, so nothing can be undeclared.
138+
const { logger, warnings } = recordingLogger();
139+
const engine = engineWith(logger);
140+
141+
engine.registerFlow('f', flowWith('decision', { condition: 'a == b', whateverElse: 1 }));
142+
143+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
144+
});
145+
146+
it('does not flag the keys of a keyValue map — those are author data', () => {
147+
// The walk descends where the schema declares structure and STOPS at a
148+
// free-form map: `filter: { status: 'stale' }` keys are data, not config keys.
149+
const { logger, warnings } = recordingLogger();
150+
const engine = engineWith(logger);
151+
152+
engine.registerFlow('f', flowWith('get_record', {
153+
objectName: 'crm_lead',
154+
filter: { status: 'stale', anythingAtAll: true },
155+
}));
156+
157+
expect(warnings.filter((w) => w.includes('unknown config key'))).toEqual([]);
158+
});
159+
});

packages/services/service-automation/src/engine.ts

Lines changed: 119 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,18 @@ import { ConnectorSchema } from '@objectstack/spec/integration';
1717
// silently returned `false`, so EVERY start-node / edge condition (record-change
1818
// `previous.*`, `budget > 100000`, …) skipped its flow. A static import binds the
1919
// engine at module load in both ESM and CJS builds.
20-
import { ExpressionEngine, validateExpression } from '@objectstack/formula';
20+
import { ExpressionEngine, validateExpression, nearestName } from '@objectstack/formula';
21+
22+
/**
23+
* The slice of a descriptor's JSON-Schema `configSchema` that the undeclared-key
24+
* walk reads (#4045). Structural only — no validation semantics.
25+
*/
26+
interface ConfigSchemaNode {
27+
type?: string;
28+
properties?: Record<string, ConfigSchemaNode>;
29+
items?: ConfigSchemaNode;
30+
additionalProperties?: unknown;
31+
}
2132
import { runIsUnscopedUserMode, flowTouchesData } from './runtime-identity.js';
2233
import { isGuardRefusal } from './guard-refusal.js';
2334

@@ -1473,6 +1484,12 @@ export class AutomationEngine implements IAutomationService {
14731484
// executeNode() already throws NO_EXECUTOR at run time for unknown types.
14741485
this.validateNodeTypes(name, parsed);
14751486

1487+
// #4045 — warn on config keys the node's descriptor does not declare
1488+
// (a `visibleIf` typo is silently accepted today). Warn-only: see
1489+
// validateNodeConfigKeys for why hard-failing would gamble on the nine
1490+
// builtins whose read-vs-declared drift has not been audited.
1491+
this.validateNodeConfigKeys(name, parsed);
1492+
14761493
// ADR-0032 §Decision 1a — parse-validate every predicate at registration,
14771494
// so a malformed condition (e.g. the #1491 `{record.x}` template-brace-in-
14781495
// CEL mistake) is a LOUD registration error with the offending source,
@@ -2717,6 +2734,107 @@ export class AutomationEngine implements IAutomationService {
27172734
}
27182735
}
27192736

2737+
/**
2738+
* Warn about node `config` keys the node type's descriptor does not declare
2739+
* (#4045 — the warn half of the unknown-key ladder).
2740+
*
2741+
* `FlowNodeSchema.config` is `z.record(z.unknown())`, so a misspelled or
2742+
* invented config key is accepted in total silence today: `visibleIf` instead
2743+
* of `visibleWhen` registers cleanly and then does nothing, which is exactly
2744+
* the failure shape that made #3528 take three passes to diagnose. The key is
2745+
* never read, so there is no runtime error to trace back — the only symptom
2746+
* is a feature that quietly does not happen.
2747+
*
2748+
* **Warn, never reject.** An undeclared key falls into three populations and
2749+
* this seam cannot yet tell them apart: an author typo (which we want to
2750+
* reject), a key the executor genuinely reads that its hand-written
2751+
* `configSchema` never declared (`notify.source` was exactly this until
2752+
* #4045 — rejecting those breaks working apps), and dead config nobody reads
2753+
* (harmless). Only 4 of the 13 schema-carrying builtins have been audited for
2754+
* the second population, so hard-failing here would gamble on the 9 that have
2755+
* not. The warning is what measures that distribution; tightening to an error
2756+
* is a later, per-key decision once the data exists, and belongs with a
2757+
* tombstone that carries the prescription (the `UNKNOWN_KEY_GUIDANCE` pattern
2758+
* in `object.zod.ts`).
2759+
*
2760+
* Soft-fail matches {@link validateNodeTypes} directly above — same function,
2761+
* same `logger.warn` channel, same reasoning about not breaking flows over a
2762+
* diagnostic. Nothing about the published `configSchema` changes, so no
2763+
* consumer (designer form generation included) sees a different shape.
2764+
*/
2765+
private validateNodeConfigKeys(flowName: string, flow: FlowParsed): void {
2766+
for (const node of flow.nodes) {
2767+
const schema = this.actionDescriptors.get(node.type)?.configSchema as ConfigSchemaNode | undefined;
2768+
// No descriptor, or a deliberately schemaless type (`decision`,
2769+
// `script`, `wait`, `subflow` publish none — see config-schemas.test)
2770+
// ⇒ nothing is declared, so nothing can be undeclared.
2771+
if (!schema) continue;
2772+
this.warnUndeclaredConfigKeys(flowName, node, schema, node.config, 'config');
2773+
}
2774+
}
2775+
2776+
/**
2777+
* Walk `value` against `schema` in lockstep, warning on keys the schema does
2778+
* not declare.
2779+
*
2780+
* Descends **only where the schema declares structure** — an object with fixed
2781+
* `properties`, or an array whose `items` do. It deliberately stops at a
2782+
* keyValue map (`additionalProperties: true` with no `properties`): those keys
2783+
* are author *data* (`filter: { status: 'stale' }`), not config keys, and
2784+
* flagging them would make the check useless noise.
2785+
*
2786+
* Descending matters rather than being thoroughness for its own sake: the
2787+
* #3528 typo class lives *inside* the `screen` field repeater, not at the top
2788+
* level. `visibleWhen` is a property of `fields[].items`, so a top-level-only
2789+
* comparison would miss `visibleIf` — the exact mistake this check exists to
2790+
* catch — while still reporting the rarer top-level ones. A test pins it.
2791+
*/
2792+
private warnUndeclaredConfigKeys(
2793+
flowName: string,
2794+
node: FlowNodeParsed,
2795+
schema: ConfigSchemaNode,
2796+
value: unknown,
2797+
path: string,
2798+
): void {
2799+
if (schema.type === 'array' && schema.items) {
2800+
if (!Array.isArray(value)) return;
2801+
value.forEach((element, index) => {
2802+
this.warnUndeclaredConfigKeys(flowName, node, schema.items!, element, `${path}[${index}]`);
2803+
});
2804+
return;
2805+
}
2806+
2807+
// A free-form map declares no properties — its keys are author data.
2808+
if (!schema.properties || schema.additionalProperties === true) return;
2809+
if (value == null || typeof value !== 'object' || Array.isArray(value)) return;
2810+
2811+
const declared = Object.keys(schema.properties);
2812+
for (const key of Object.keys(value as Record<string, unknown>)) {
2813+
const child = schema.properties[key];
2814+
if (child) {
2815+
this.warnUndeclaredConfigKeys(flowName, node, child, (value as Record<string, unknown>)[key], `${path}.${key}`);
2816+
continue;
2817+
}
2818+
// The declared set is printed ALWAYS, not only as a fallback when the
2819+
// edit-distance heuristic misses. It misses more than you would
2820+
// expect: `visibleIf` → `visibleWhen` is distance 4 against a
2821+
// threshold of 3, so the very typo this check exists to catch gets no
2822+
// suggestion. Loosening `nearestName` is the wrong fix — it is shared
2823+
// with the unknown-field and unknown-role diagnostics, where a looser
2824+
// threshold means confidently wrong suggestions over hundreds of
2825+
// candidates. A config object declares at most a dozen keys, so simply
2826+
// listing them is both cheap and complete, and the suggestion becomes
2827+
// a bonus for the cases it does catch.
2828+
const suggestion = nearestName(key, declared);
2829+
this.logger.warn(
2830+
`[flow '${flowName}'] node '${node.id}' (${node.type}): unknown config key \`${key}\` at ${path}.${key}` +
2831+
(suggestion ? ` — did you mean \`${suggestion}\`?` : '') +
2832+
` It is not declared by this node type's configSchema, so nothing reads it.` +
2833+
` Declared here: ${declared.join(', ')}.`,
2834+
);
2835+
}
2836+
}
2837+
27202838
/**
27212839
* ADR-0032 §Decision 1a — parse-validate every predicate in the flow at
27222840
* registration. Predicates are bare CEL; this catches the #1491 class

0 commit comments

Comments
 (0)