Skip to content

Commit 230358c

Browse files
os-zhuangclaude
andauthored
feat(mcp): validate_expression tool — validate CEL against a schema before authoring (#1928) (#3203)
Adds an agent-callable MCP tool that runs the same build-time expression checks as objectstack build, so an AI can validate a formula / predicate / flow condition while authoring instead of shipping one that silently evaluates to null. Given { objectName, expression, site? } it resolves the object's real schema (field names + types via the principal-bound describeObject bridge) and returns errors (bare refs, unknown fields/functions), warnings (text/boolean in arithmetic, date-equality), the fields/functions/roots in scope for self-correction, and an inferred type for formula sites. site (formula | validation | flow_condition | template, default formula) maps to the validator's role + scope. Read-only, data:read-gated, fail-closed on sys_*. mcp gains a @objectstack/formula dependency (acyclic; formula is a leaf). Tests: mcp 80 (+8 tool cases; updated the 3 tool-surface assertions + SKILL.md drift guard). Build/DTS clean. Claude-Session: https://claude.ai/code/session_01Hnji7EEYR2mGt6pY53a8Bm Co-authored-by: Claude <noreply@anthropic.com>
1 parent 8a39022 commit 230358c

9 files changed

Lines changed: 250 additions & 1 deletion
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
---
2+
"@objectstack/mcp": minor
3+
---
4+
5+
feat(mcp): `validate_expression` tool — validate a CEL expression against a schema before authoring (#1928)
6+
7+
Adds an agent-callable MCP tool that runs the same build-time expression checks
8+
as `objectstack build`, so an AI can validate a formula / predicate / flow
9+
condition **while authoring** instead of shipping one that silently evaluates to
10+
`null`. Given `{ objectName, expression, site? }` it resolves the object's real
11+
schema (field names + types, via the principal-bound `describeObject` bridge)
12+
and returns:
13+
14+
- **errors** — bare field refs (`amount``record.amount`), unknown fields
15+
(with a did-you-mean), unknown functions;
16+
- **warnings** — text/boolean fields misused in arithmetic, date-equality
17+
pitfalls;
18+
- **inScope** — the fields, stdlib functions, and namespace roots available, so
19+
the model can self-correct;
20+
- **inferredType** for a `formula` site.
21+
22+
`site` (`formula` | `validation` | `flow_condition` | `template`, default
23+
`formula`) maps to the validator's role + scope — `flow_condition` binds fields
24+
bare, the rest bind `record.<field>`. Read-only, gated by the `data:read` OAuth
25+
scope, and fail-closed on `sys_*` objects like the other schema tools. This is
26+
the authoring-time surface the guardrail series (#1928) always pointed at;
27+
`@objectstack/mcp` gains a `@objectstack/formula` dependency (acyclic; formula is
28+
a leaf).

packages/mcp/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ the open framework.
105105
// Object data (RLS-enforced as the caller)
106106
'list_objects' // List objects (system sys_* objects hidden by default)
107107
'describe_object' // Object schema: fields + features
108+
'validate_expression' // Check a CEL expression against a schema before authoring it
108109
'query_records' // Filter / sort / paginate
109110
'get_record' // Fetch one by id
110111
'create_record' / 'update_record' / 'delete_record'

packages/mcp/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@
2020
"dependencies": {
2121
"@modelcontextprotocol/sdk": "^1.29.0",
2222
"@objectstack/core": "workspace:*",
23+
"@objectstack/formula": "workspace:*",
2324
"@objectstack/spec": "workspace:*",
2425
"@objectstack/types": "workspace:*",
2526
"zod": "^4.4.3"

packages/mcp/src/mcp-http-tools.scopes.test.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ import {
2222
import { MCPServerRuntime } from './mcp-server-runtime.js';
2323
import type { McpActionBridge, McpDataBridge } from './mcp-http-tools.js';
2424

25-
const READ_TOOLS = ['list_objects', 'describe_object', 'query_records', 'get_record', 'aggregate_records'];
25+
const READ_TOOLS = ['list_objects', 'describe_object', 'validate_expression', 'query_records', 'get_record', 'aggregate_records'];
2626
const WRITE_TOOLS = ['create_record', 'update_record', 'delete_record'];
2727
const ACTION_TOOLS = ['list_actions', 'run_action'];
2828

packages/mcp/src/mcp-http-tools.ts

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,12 @@ import {
2727
MCP_OAUTH_SCOPE_DATA_WRITE,
2828
MCP_OAUTH_SCOPE_ACTIONS,
2929
} from '@objectstack/spec/ai';
30+
import {
31+
validateExpression,
32+
introspectScope,
33+
inferExpressionType,
34+
type FieldRole,
35+
} from '@objectstack/formula';
3036

3137
export interface McpObjectSummary {
3238
name: string;
@@ -189,6 +195,21 @@ function jsonText(value: unknown): string {
189195
}
190196
}
191197

198+
/**
199+
* Authoring site → the `(role, scope)` the shared validator uses. A `formula`
200+
* field is a `value` expression bound to the `record` namespace; a `validation`
201+
* rule is a `record`-scoped `predicate`; a `flow_condition` is a `predicate`
202+
* whose fields are flattened to top level; a `template` is a text template.
203+
* Exposing a single friendly `site` keeps the tool aligned with how an author
204+
* thinks about *where* the expression goes.
205+
*/
206+
const VALIDATE_SITE_MAP: Record<string, { role: FieldRole; scope: 'record' | 'flattened' }> = {
207+
formula: { role: 'value', scope: 'record' },
208+
validation: { role: 'predicate', scope: 'record' },
209+
flow_condition: { role: 'predicate', scope: 'flattened' },
210+
template: { role: 'template', scope: 'record' },
211+
};
212+
192213
/**
193214
* Register the object-CRUD tool set on a fresh per-request {@link McpServer}.
194215
* All execution is delegated to `bridge`, which is bound to the caller's
@@ -258,6 +279,69 @@ export function registerObjectTools(
258279
},
259280
);
260281

282+
// Validate a CEL expression against a real object schema BEFORE it is
283+
// authored into metadata — the same checks `objectstack build` runs, so an
284+
// agent gets a build-accurate verdict plus the fields/functions in scope to
285+
// self-correct, instead of shipping a formula that silently evaluates to
286+
// `null` (#1928). Read-only (schema introspection); no data is touched.
287+
server.registerTool(
288+
'validate_expression',
289+
{
290+
description:
291+
'Validate a CEL expression against an object\'s schema before authoring it into metadata. Returns ' +
292+
'build-time errors (bare field refs, unknown fields, unknown functions) and advisory warnings ' +
293+
'(text/boolean fields misused in arithmetic, date-equality pitfalls), plus the fields and stdlib ' +
294+
'functions in scope so you can self-correct. `site` says where the expression will live: a `formula` ' +
295+
'field, a `validation`/predicate, or a `flow_condition` (fields are bound bare in flow conditions).',
296+
inputSchema: {
297+
objectName: z.string().describe('The object/table the expression is authored against, e.g. "task"'),
298+
expression: z.string().describe('The CEL expression to validate, e.g. "record.amount / 100"'),
299+
site: z
300+
.enum(['formula', 'validation', 'flow_condition', 'template'])
301+
.optional()
302+
.describe(
303+
'Where the expression will live. formula/validation bind `record.<field>`; flow_condition binds fields bare. Default: formula.',
304+
),
305+
},
306+
annotations: { readOnlyHint: true, destructiveHint: false, openWorldHint: false },
307+
},
308+
async ({ objectName, expression, site }) => {
309+
const bad = guard(objectName);
310+
if (bad) return errorResult(bad);
311+
try {
312+
const def = (await bridge.describeObject(objectName)) as { fields?: unknown } | null;
313+
if (!def) return errorResult(`Object "${objectName}" not found`);
314+
const fieldDefs = Array.isArray(def.fields) ? (def.fields as Array<Record<string, unknown>>) : [];
315+
const fields: string[] = [];
316+
const fieldTypes: Record<string, string> = {};
317+
for (const f of fieldDefs) {
318+
if (typeof f?.name !== 'string') continue;
319+
fields.push(f.name);
320+
if (typeof f?.type === 'string') fieldTypes[f.name] = f.type;
321+
}
322+
const { role, scope } = VALIDATE_SITE_MAP[site ?? 'formula'];
323+
const hint = { objectName, fields, fieldTypes, scope } as const;
324+
const result = validateExpression(role, expression, hint);
325+
const inScope = introspectScope(role, hint);
326+
const inferredType = role === 'value' ? inferExpressionType(expression, hint) : undefined;
327+
return textResult({
328+
ok: result.ok,
329+
errors: result.errors,
330+
warnings: result.warnings,
331+
...(inferredType ? { inferredType } : {}),
332+
inScope: {
333+
dialect: inScope.dialect,
334+
roots: inScope.roots,
335+
fields: inScope.fields,
336+
functions: inScope.functions,
337+
},
338+
});
339+
} catch (err) {
340+
return errorResult(messageOf(err));
341+
}
342+
},
343+
);
344+
261345
server.registerTool(
262346
'query_records',
263347
{

packages/mcp/src/mcp-server-runtime.http.test.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@ describe('MCPServerRuntime.handleHttpRequest (Streamable HTTP)', () => {
9797
'list_objects',
9898
'query_records',
9999
'update_record',
100+
'validate_expression',
100101
].sort(),
101102
);
102103
});
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { describe, it, expect, beforeEach } from 'vitest';
4+
5+
import { MCPServerRuntime } from './mcp-server-runtime.js';
6+
import type { McpDataBridge } from './mcp-http-tools.js';
7+
8+
/**
9+
* A data bridge whose `describeObject` returns a realistic schema (field names
10+
* AND types), so the `validate_expression` tool can run the full build-time
11+
* check — including the #1928 type-soundness warning that needs field types.
12+
*/
13+
function makeBridge(): McpDataBridge {
14+
return {
15+
async listObjects() {
16+
return [{ name: 'crm_opportunity', label: 'Opportunity', fieldCount: 4 }];
17+
},
18+
async describeObject(name: string) {
19+
if (name !== 'crm_opportunity') return null;
20+
return {
21+
name: 'crm_opportunity',
22+
label: 'Opportunity',
23+
fields: [
24+
{ name: 'amount', type: 'currency', label: 'Amount', required: false },
25+
{ name: 'probability', type: 'percent', label: 'Probability', required: false },
26+
{ name: 'title', type: 'text', label: 'Title', required: true },
27+
{ name: 'is_active', type: 'boolean', label: 'Active', required: false },
28+
{ name: 'stage', type: 'select', label: 'Stage', required: false },
29+
],
30+
};
31+
},
32+
async query() { return { records: [] }; },
33+
async get() { return null; },
34+
async create() { return {}; },
35+
async update() { return {}; },
36+
async remove() { return {}; },
37+
};
38+
}
39+
40+
function mcpRequest(body: unknown): Request {
41+
return new Request('http://localhost/mcp', {
42+
method: 'POST',
43+
headers: { 'content-type': 'application/json', accept: 'application/json, text/event-stream' },
44+
body: JSON.stringify(body),
45+
});
46+
}
47+
48+
async function call(runtime: MCPServerRuntime, body: unknown, bridge?: unknown) {
49+
const res = await runtime.handleHttpRequest(mcpRequest(body), { bridge, parsedBody: body } as never);
50+
const json = res.status === 202 ? null : await res.json();
51+
return { status: res.status, json };
52+
}
53+
54+
const toolsCall = (id: number, name: string, args: Record<string, unknown>) => ({
55+
jsonrpc: '2.0', id, method: 'tools/call', params: { name, arguments: args },
56+
});
57+
58+
async function validate(runtime: MCPServerRuntime, bridge: McpDataBridge, args: Record<string, unknown>) {
59+
const { json } = await call(runtime, toolsCall(1, 'validate_expression', args), bridge);
60+
return JSON.parse(json.result.content[0].text);
61+
}
62+
63+
describe('validate_expression MCP tool (#1928)', () => {
64+
let runtime: MCPServerRuntime;
65+
let bridge: McpDataBridge;
66+
67+
beforeEach(() => {
68+
runtime = new MCPServerRuntime({ name: 'objectstack-test', version: '9.9.9' });
69+
bridge = makeBridge();
70+
});
71+
72+
it('registers as a read-only tool', async () => {
73+
const { json } = await call(runtime, { jsonrpc: '2.0', id: 1, method: 'tools/list' }, bridge);
74+
const byName = Object.fromEntries(json.result.tools.map((t: any) => [t.name, t]));
75+
expect(byName.validate_expression).toBeDefined();
76+
expect(byName.validate_expression.annotations.readOnlyHint).toBe(true);
77+
});
78+
79+
it('accepts a sound formula and reports its inferred type + in-scope context', async () => {
80+
const r = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'record.amount / 100', site: 'formula' });
81+
expect(r.ok).toBe(true);
82+
expect(r.errors).toHaveLength(0);
83+
expect(r.warnings).toHaveLength(0); // currency → dyn, so int-literal arithmetic is fine
84+
expect(r.inScope.fields).toContain('amount');
85+
expect(r.inScope.functions).toContain('daysBetween');
86+
});
87+
88+
it('flags a bare field reference in a record-scoped formula (error)', async () => {
89+
const r = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'amount > 100', site: 'formula' });
90+
expect(r.ok).toBe(false);
91+
expect(r.errors[0].message).toMatch(/bare reference `amount`|record\.amount/);
92+
});
93+
94+
it('flags an unknown field with a did-you-mean (error)', async () => {
95+
const r = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'record.amont > 100', site: 'validation' });
96+
expect(r.ok).toBe(false);
97+
expect(r.errors[0].message).toMatch(/unknown field `amont`/);
98+
});
99+
100+
it('warns (does not error) on a text field misused in arithmetic — tier 4', async () => {
101+
const r = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'record.title * 2', site: 'formula' });
102+
expect(r.ok).toBe(true);
103+
expect(r.warnings.some((w: any) => /type mismatch/i.test(w.message))).toBe(true);
104+
});
105+
106+
it('validates a flattened flow condition (bare fields correct; type-soundness still applies)', async () => {
107+
const ok = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'stage == "won" && amount > 1000', site: 'flow_condition' });
108+
expect(ok.ok).toBe(true);
109+
expect(ok.warnings).toHaveLength(0);
110+
const warn = await validate(runtime, bridge, { objectName: 'crm_opportunity', expression: 'title * 2 > 10', site: 'flow_condition' });
111+
expect(warn.warnings.some((w: any) => /type mismatch/i.test(w.message))).toBe(true);
112+
});
113+
114+
it('errors clearly when the object does not exist', async () => {
115+
const { json } = await call(runtime, toolsCall(1, 'validate_expression', { objectName: 'nope', expression: 'record.x > 1' }), bridge);
116+
expect(json.result.isError).toBe(true);
117+
expect(json.result.content[0].text).toMatch(/not found/i);
118+
});
119+
120+
it('refuses system objects (fail-closed guard)', async () => {
121+
const { json } = await call(runtime, toolsCall(1, 'validate_expression', { objectName: 'sys_user', expression: 'record.x > 1' }), bridge);
122+
expect(json.result.isError).toBe(true);
123+
expect(json.result.content[0].text).toMatch(/system object/i);
124+
});
125+
});

packages/mcp/src/skill.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,12 @@ create/update payload.
129129
130130
- **list_objects()** — list available objects (system \`sys_*\` objects are hidden).
131131
- **describe_object({ objectName })** — an object's fields and features.
132+
- **validate_expression({ objectName, expression, site? })** — check a CEL
133+
expression against an object's schema *before* you author it into a formula,
134+
validation, or flow condition. Returns build-time errors (bare field refs,
135+
unknown fields/functions) and advisory warnings (text/boolean fields misused
136+
in arithmetic), plus the fields and stdlib functions in scope. \`site\` is
137+
\`formula\` | \`validation\` | \`flow_condition\` | \`template\` (default \`formula\`).
132138
- **query_records({ objectName, where?, fields?, limit?, offset?, orderBy? })** —
133139
read records. \`where\` is a field→value match, e.g. \`{ "status": "open" }\`.
134140
Results are page-capped; use \`limit\`/\`offset\` to page.

pnpm-lock.yaml

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)