Skip to content

Commit e890245

Browse files
committed
fix(runtime): honor hook body timeoutMs so nested cross-object writes aren't clamped to 250ms (#1867)
Nested cross-object writes from a hook (`ctx.api.object('parent').update(...)` from a child's afterInsert/afterUpdate) once crashed the QuickJS sandbox with `memory access out of bounds` — the old single-suspended-asyncify host-call model could not unwind the WASM stack twice. That crash is already fixed on main by the deferred-promise + pump host-call model (host calls no longer asyncify; each invocation gets its own isolated VM), so any depth of nested re-entrancy now composes safely. The remaining reliability blocker was the timeout: `resolveTimeout` folded the engine default (250ms for hooks) straight into `Math.min(...)`, so it always dominated. A hook body that declared a larger `timeoutMs` (the spec allows up to 30_000ms via `ScriptBody.timeoutMs`) to give a legitimate nested-write rollup room to settle was silently clamped back to 250ms and killed mid-flight — a declared-but-unenforced knob (AGENTS.md Prime Directive #10) that pushed authors toward denormalized hand-maintained rollups. The engine default is now a FALLBACK used only when no explicit timeout is supplied, not a hard ceiling. An explicit `body.timeoutMs` (and/or an enclosing hook/action timeout via opts) is honored; when both are present the smaller wins. Unspecified bodies still get the 250ms hook / 5000ms action default, and a body may still lower its own timeout below the default. Tests: - quickjs-runner.test.ts — nested sandbox re-entrancy (single, 4-level chain, concurrent fan-out) does not crash; timeout resolution honors a body's declared timeoutMs above the default, still applies the default when unspecified, and still lets a body lower it. - nested-write.integration.test.ts — real ObjectQL engine + sandbox: a child afterInsert/afterUpdate hook rolls its total up to the parent (parent also has a hook → real nested VM) without crashing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BZguyAaQbyUpwMZ2gMLaAP
1 parent 06cb319 commit e890245

4 files changed

Lines changed: 366 additions & 2 deletions

File tree

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
---
2+
'@objectstack/runtime': patch
3+
---
4+
5+
fix(runtime): honor a hook body's declared `timeoutMs` so nested cross-object writes aren't clamped to 250ms (#1867)
6+
7+
Hook bodies run in the QuickJS sandbox with a default 250ms timeout. The runner
8+
folded that engine default straight into `Math.min(...)` when resolving the
9+
effective timeout, so it *always* dominated for hooks: a body that declared a
10+
larger `timeoutMs` (the spec permits up to 30_000ms — `ScriptBody.timeoutMs`) to
11+
give a legitimate nested write — "when a child changes, update the parent" —
12+
room to settle was silently clamped back to 250ms and killed mid-flight. The
13+
declared knob was never enforced.
14+
15+
The engine default is now a FALLBACK used only when no explicit timeout is
16+
supplied, not a hard ceiling. An explicit `body.timeoutMs` (and/or an enclosing
17+
hook/action timeout) is honored; when both are present the smaller wins. Bodies
18+
that declare nothing still get the 250ms hook / 5000ms action default, and a
19+
body may still LOWER its own timeout below the default.
20+
21+
This clears the last reliability blocker for nested cross-object writes from
22+
hooks — the sandbox crash itself (`memory access out of bounds`) was already
23+
fixed by the deferred-promise host-call model — so header/rollup fields no
24+
longer need denormalized, hand-maintained workarounds.
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* End-to-end regression for #1867 — nested cross-object writes from a hook.
5+
*
6+
* A child object's `afterInsert` / `afterUpdate` hook issues an engine write to
7+
* its parent (`ctx.api.object('parent').update(...)`). The parent *also* has a
8+
* hook, so the child's nested write fires a SECOND sandbox VM while the child's
9+
* hook is still in flight — the exact re-entrancy that used to crash the process
10+
* with `memory access out of bounds` under the old single-suspended-asyncify
11+
* model. This wires a REAL {@link ObjectQL} engine (in-memory driver) to the
12+
* real {@link QuickJSScriptRunner} through {@link hookBodyRunnerFactory}, so the
13+
* whole hook → sandbox → nested-write → nested-hook path is exercised.
14+
*
15+
* This is the "when a child changes, update the parent" rollup the issue says
16+
* authors could not write; it must complete without a crash and land the
17+
* correct denormalized total on the parent.
18+
*/
19+
20+
import { describe, it, expect, beforeEach } from 'vitest';
21+
import { ObjectQL, bindHooksToEngine } from '@objectstack/objectql';
22+
import { hookBodyRunnerFactory } from './body-runner.js';
23+
import { QuickJSScriptRunner } from './quickjs-runner.js';
24+
25+
const parent = {
26+
name: 'parent',
27+
label: 'Parent',
28+
fields: {
29+
id: { name: 'id', type: 'text' as const, primaryKey: true },
30+
total_amount: { name: 'total_amount', type: 'number' as const },
31+
child_count: { name: 'child_count', type: 'number' as const },
32+
},
33+
};
34+
const child = {
35+
name: 'child',
36+
label: 'Child',
37+
fields: {
38+
id: { name: 'id', type: 'text' as const, primaryKey: true },
39+
amount: { name: 'amount', type: 'number' as const },
40+
parent_id: { name: 'parent_id', type: 'text' as const },
41+
},
42+
};
43+
44+
function makeMemoryDriver() {
45+
const stores = new Map<string, Map<string, Record<string, unknown>>>();
46+
const storeFor = (o: string) => {
47+
let s = stores.get(o);
48+
if (!s) { s = new Map(); stores.set(o, s); }
49+
return s;
50+
};
51+
let nextId = 0;
52+
const matches = (row: Record<string, unknown>, where: any): boolean => {
53+
if (!where || typeof where !== 'object') return true;
54+
for (const [k, v] of Object.entries(where)) {
55+
if (k.startsWith('$')) continue;
56+
const exp = (v && typeof v === 'object' && '$eq' in (v as any)) ? (v as any).$eq : v;
57+
if ((row[k] ?? null) !== (exp ?? null)) return false;
58+
}
59+
return true;
60+
};
61+
const driver: any = {
62+
name: 'memory', version: '0.0.0', supports: {},
63+
async connect() {}, async disconnect() {}, async checkHealth() { return true; }, async execute() { return null; },
64+
async find(o: string, ast: any) { return Array.from(storeFor(o).values()).filter((r) => matches(r, ast?.where)); },
65+
findStream() { throw new Error('ns'); },
66+
async findOne(o: string, ast: any) { for (const r of storeFor(o).values()) if (matches(r, ast?.where)) return r; return null; },
67+
async create(o: string, data: Record<string, unknown>) {
68+
nextId += 1; const id = (data.id as string) ?? `r_${nextId}`; const row = { ...data, id }; storeFor(o).set(id, row); return row;
69+
},
70+
async update(o: string, id: string, data: Record<string, unknown>) {
71+
const s = storeFor(o); const cur = s.get(id); if (!cur) throw new Error(`nf ${o}/${id}`);
72+
const up = { ...cur, ...data, id }; s.set(id, up); return up;
73+
},
74+
async upsert(o: string, data: Record<string, unknown>) { const id = data.id as string | undefined; return id && storeFor(o).has(id) ? this.update(o, id, data) : this.create(o, data); },
75+
async delete(o: string, id: string) { return storeFor(o).delete(id); },
76+
async count(o: string, ast: any) { return (await this.find(o, ast)).length; },
77+
async bulkCreate(o: string, rows: Record<string, unknown>[]) { return Promise.all(rows.map((r) => this.create(o, r))); },
78+
async bulkUpdate() { return []; }, async bulkDelete() {},
79+
async beginTransaction() { return { commit: async () => {}, rollback: async () => {} }; }, async commit() {}, async rollback() {},
80+
};
81+
return { driver, stores };
82+
}
83+
84+
describe('#1867 nested cross-object write from a hook (real engine + sandbox)', () => {
85+
let engine: ObjectQL;
86+
87+
beforeEach(async () => {
88+
engine = new ObjectQL();
89+
const { driver } = makeMemoryDriver();
90+
engine.registerDriver(driver, true);
91+
await engine.init();
92+
for (const o of [parent, child]) engine.registry.registerObject(o as any);
93+
engine.setDefaultBodyRunner(
94+
hookBodyRunnerFactory(new QuickJSScriptRunner(), { ql: engine, appId: 'test' }),
95+
);
96+
});
97+
98+
it('a child afterInsert hook writes the parent rollup without crashing (parent has its own hook → real nested VM)', async () => {
99+
bindHooksToEngine(
100+
engine,
101+
[
102+
{
103+
name: 'rollup_parent_total',
104+
object: 'child',
105+
events: ['afterInsert', 'afterUpdate'],
106+
body: {
107+
language: 'js',
108+
source: `
109+
// On update the patch may omit parent_id; fall back to the
110+
// pre-write record so the rollup targets the right parent.
111+
const pid = ctx.input.parent_id || (ctx.previous && ctx.previous.parent_id);
112+
const rows = await ctx.api.object('child').find({ where: { parent_id: pid } });
113+
const total = rows.reduce((s, r) => s + (r.amount || 0), 0);
114+
await ctx.api.object('parent').update({ id: pid, total_amount: total, child_count: rows.length });
115+
`,
116+
capabilities: ['api.read', 'api.write'],
117+
},
118+
} as any,
119+
{
120+
name: 'parent_touch_marker',
121+
object: 'parent',
122+
events: ['afterUpdate'],
123+
// Trivial body — its mere presence forces a second sandbox VM to run
124+
// (re-entrant) while the child's hook VM is still suspended.
125+
body: { language: 'js', source: "return { seen: ctx.input.id };", capabilities: [] },
126+
} as any,
127+
],
128+
{ packageId: 'test' },
129+
);
130+
131+
const p = await engine.insert('parent', { id: 'p1', total_amount: 0, child_count: 0 });
132+
await engine.insert('child', { id: 'c1', amount: 100, parent_id: p.id });
133+
await engine.insert('child', { id: 'c2', amount: 50, parent_id: p.id });
134+
135+
const parentRow = (await engine.findOne('parent', { where: { id: 'p1' } })) as any;
136+
expect(parentRow.total_amount).toBe(150);
137+
expect(parentRow.child_count).toBe(2);
138+
}, 20000);
139+
140+
it('re-rolls up on child update (afterUpdate nested write)', async () => {
141+
bindHooksToEngine(
142+
engine,
143+
[
144+
{
145+
name: 'rollup_parent_total',
146+
object: 'child',
147+
events: ['afterInsert', 'afterUpdate'],
148+
body: {
149+
language: 'js',
150+
source: `
151+
// On update the patch may omit parent_id; fall back to the
152+
// pre-write record so the rollup targets the right parent.
153+
const pid = ctx.input.parent_id || (ctx.previous && ctx.previous.parent_id);
154+
const rows = await ctx.api.object('child').find({ where: { parent_id: pid } });
155+
const total = rows.reduce((s, r) => s + (r.amount || 0), 0);
156+
await ctx.api.object('parent').update({ id: pid, total_amount: total, child_count: rows.length });
157+
`,
158+
capabilities: ['api.read', 'api.write'],
159+
},
160+
} as any,
161+
],
162+
{ packageId: 'test' },
163+
);
164+
165+
await engine.insert('parent', { id: 'p1', total_amount: 0, child_count: 0 });
166+
await engine.insert('child', { id: 'c1', amount: 100, parent_id: 'p1' });
167+
expect(((await engine.findOne('parent', { where: { id: 'p1' } })) as any).total_amount).toBe(100);
168+
169+
await engine.update('child', { id: 'c1', amount: 250, parent_id: 'p1' });
170+
expect(((await engine.findOne('parent', { where: { id: 'p1' } })) as any).total_amount).toBe(250);
171+
}, 20000);
172+
});

packages/runtime/src/sandbox/quickjs-runner.test.ts

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,156 @@ describe('QuickJSScriptRunner — async host APIs', () => {
264264
});
265265
});
266266

267+
// ---------------------------------------------------------------------------
268+
// Nested cross-object writes (#1867).
269+
//
270+
// A hook body that issues an engine write (`ctx.api.object('parent').update`)
271+
// re-enters the sandbox: the host-side write fires the *parent's* hook, which
272+
// runs its own body inside a fresh VM while the child's hook is still in flight.
273+
// The old asyncify host-call model crashed here ("memory access out of bounds"
274+
// — the stack cannot be unwound twice). The deferred-promise + pump model must
275+
// compose any depth of nesting safely.
276+
// ---------------------------------------------------------------------------
277+
describe('QuickJSScriptRunner — nested sandbox re-entrancy (#1867)', () => {
278+
it('a host write that re-invokes the runner (parent hook) does not crash and returns correctly', async () => {
279+
// The parent's afterUpdate hook body, run when the child writes the parent.
280+
const parentBody = {
281+
language: 'js' as const,
282+
source: 'return { parentTouched: true, doubled: (ctx.input.n || 0) * 2 };',
283+
capabilities: [] as const,
284+
};
285+
const api = {
286+
object: (_n: string) => ({
287+
update: async (patch: Record<string, unknown>) => {
288+
// Re-enter the sandbox exactly as the engine does when the parent
289+
// write fires the parent's own hook body.
290+
const nested = await runner.run(
291+
parentBody,
292+
{ input: { n: 21 } } as ScriptContext,
293+
{ origin: { kind: 'hook', name: 'parent_hook' } },
294+
);
295+
return { updated: patch, nested: nested.value };
296+
},
297+
}),
298+
};
299+
const r = await runner.run(
300+
{
301+
language: 'js',
302+
source: "return await ctx.api.object('parent').update({ total: ctx.input.amount });",
303+
capabilities: ['api.write'],
304+
},
305+
ctx({ input: { amount: 100 }, api }),
306+
{ origin: { kind: 'hook', name: 'child_hook' } },
307+
);
308+
expect(r.value).toEqual({
309+
updated: { total: 100 },
310+
nested: { parentTouched: true, doubled: 42 },
311+
});
312+
}, 15000);
313+
314+
it('survives a multi-level nested write chain (child → parent → grandparent → …)', async () => {
315+
const makeApi = (depth: number): any => ({
316+
object: () => ({
317+
update: async () => {
318+
if (depth <= 0) return { leaf: true };
319+
const nested = await runner.run(
320+
{ language: 'js', source: "return await ctx.api.object('x').update({});", capabilities: ['api.write'] },
321+
{ input: {}, api: makeApi(depth - 1) } as ScriptContext,
322+
{ origin: { kind: 'hook', name: `lvl${depth}` } },
323+
);
324+
return { depth, nested: nested.value };
325+
},
326+
}),
327+
});
328+
const r = await runner.run(
329+
{ language: 'js', source: "return await ctx.api.object('x').update({});", capabilities: ['api.write'] },
330+
{ input: {}, api: makeApi(4) } as ScriptContext,
331+
{ origin: { kind: 'hook', name: 'child' }, timeoutMs: 10000 },
332+
);
333+
// Four levels of nesting resolve without a WASM crash.
334+
expect((r.value as any).nested.nested.nested.nested).toEqual({ leaf: true });
335+
}, 20000);
336+
337+
it('runs concurrent nested invocations (fan-out) without cross-VM corruption', async () => {
338+
const leaf = { language: 'js' as const, source: 'return { leaf: true };', capabilities: [] as const };
339+
const api: any = {
340+
object: () => ({
341+
update: async () => {
342+
const [a, b, c] = await Promise.all([
343+
runner.run(leaf, { input: {} } as ScriptContext, { origin: { kind: 'hook', name: 'p1' } }),
344+
runner.run(leaf, { input: {} } as ScriptContext, { origin: { kind: 'hook', name: 'p2' } }),
345+
runner.run(leaf, { input: {} } as ScriptContext, { origin: { kind: 'hook', name: 'p3' } }),
346+
]);
347+
return { a: a.value, b: b.value, c: c.value };
348+
},
349+
}),
350+
};
351+
const r = await runner.run(
352+
{ language: 'js', source: "return await ctx.api.object('x').update({});", capabilities: ['api.write'] },
353+
{ input: {}, api } as ScriptContext,
354+
{ origin: { kind: 'hook', name: 'child' }, timeoutMs: 10000 },
355+
);
356+
expect(r.value).toEqual({ a: { leaf: true }, b: { leaf: true }, c: { leaf: true } });
357+
}, 20000);
358+
});
359+
360+
// ---------------------------------------------------------------------------
361+
// Timeout resolution (#1867). The engine default is a FALLBACK, not a hard
362+
// ceiling: a hook body may declare a larger `timeoutMs` (spec allows ≤30s) so a
363+
// legitimate nested-write rollup has room to settle instead of being clamped to
364+
// the 250ms hook default and killed mid-flight.
365+
// ---------------------------------------------------------------------------
366+
describe('QuickJSScriptRunner — timeout resolution honors body.timeoutMs (#1867)', () => {
367+
it('honors a hook body timeoutMs above the 250ms hook default', async () => {
368+
// Host call settles at ~600ms — comfortably past the old 250ms hook cap but
369+
// within the body's declared 5000ms budget. Must resolve, not time out.
370+
const api = {
371+
object: () => ({
372+
update: async () => {
373+
await new Promise<void>((resolve) => setTimeout(resolve, 600));
374+
return { ok: true };
375+
},
376+
}),
377+
};
378+
const r = await runner.runScript(
379+
{
380+
language: 'js',
381+
source: "return await ctx.api.object('x').update({});",
382+
capabilities: ['api.write'],
383+
timeoutMs: 5000,
384+
},
385+
ctx({ api }),
386+
hookOpts, // hook origin → old code hard-capped at 250ms
387+
);
388+
expect(r.value).toEqual({ ok: true });
389+
}, 10000);
390+
391+
it('still applies the 250ms hook default when the body declares no timeoutMs', async () => {
392+
const api = { object: () => ({ update: () => new Promise<never>(() => {}) }) };
393+
const start = Date.now();
394+
await expect(
395+
runner.runScript(
396+
{ language: 'js', source: "return await ctx.api.object('x').update({});", capabilities: ['api.write'] },
397+
ctx({ api }),
398+
hookOpts,
399+
),
400+
).rejects.toThrow(/timeout/i);
401+
// Fired near the 250ms default, not some larger value.
402+
expect(Date.now() - start).toBeLessThan(2000);
403+
}, 10000);
404+
405+
it('lets a hook body LOWER its timeout below the default', async () => {
406+
const api = { object: () => ({ update: () => new Promise<never>(() => {}) }) };
407+
await expect(
408+
runner.runScript(
409+
{ language: 'js', source: "return await ctx.api.object('x').update({});", capabilities: ['api.write'], timeoutMs: 50 },
410+
ctx({ api }),
411+
hookOpts,
412+
),
413+
).rejects.toThrow(/timeout/i);
414+
}, 10000);
415+
});
416+
267417
describe('QuickJSScriptRunner — long-running async host work (pump budget)', () => {
268418
// Regression: an action's single `ctx.api.update(...)` can synchronously drive
269419
// a large amount of awaited host work — e.g. a record-change flow that the

packages/runtime/src/sandbox/quickjs-runner.ts

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -104,10 +104,28 @@ export class QuickJSScriptRunner implements ScriptRunner {
104104
/* no-op — runtimes are per-invocation in v1 */
105105
}
106106

107-
/** Pick the smallest of body / opts / engine-default. */
107+
/**
108+
* Resolve the effective per-invocation timeout.
109+
*
110+
* The engine default (`hookTimeoutMs` / `actionTimeoutMs`) is a FALLBACK used
111+
* only when the caller supplies no explicit timeout — it is NOT a hard
112+
* ceiling. Whichever explicit timeout the caller *did* supply wins: the body's
113+
* own `timeoutMs` (the spec permits up to 30_000ms — see `ScriptBody.timeoutMs`)
114+
* and/or an enclosing hook/action timeout passed via `opts.timeoutMs`; when
115+
* both are present the smaller wins, matching the spec's "smaller of this and
116+
* the enclosing hook/action timeout wins".
117+
*
118+
* Previously the default was folded straight into `Math.min(...)`, so for
119+
* hooks — whose default is 250ms — it *always* dominated: a body that declared
120+
* `timeoutMs: 5000` to give a legitimate nested cross-object write room to
121+
* settle was silently clamped back to 250ms and killed mid-flight. That made
122+
* the spec's `ScriptBody.timeoutMs` a declared-but-unenforced knob for hooks
123+
* and pushed template authors toward denormalized rollup workarounds (#1867).
124+
*/
108125
private resolveTimeout(opts: ScriptRunOptions, bodyTimeoutMs: number | undefined): number {
109126
const def = opts.origin.kind === 'hook' ? this.opts.hookTimeoutMs : this.opts.actionTimeoutMs;
110-
return Math.min(...[def, opts.timeoutMs, bodyTimeoutMs].filter((n): n is number => typeof n === 'number'));
127+
const explicit = [opts.timeoutMs, bodyTimeoutMs].filter((n): n is number => typeof n === 'number');
128+
return explicit.length > 0 ? Math.min(...explicit) : def;
111129
}
112130

113131
private async execute(args: {

0 commit comments

Comments
 (0)