Skip to content

Commit c2d9098

Browse files
os-zhuangclaude
andauthored
feat(rest/protocol): extend droppedFields write-observability to bulk paths + client SDK (#3455) (#3468)
* feat(rest/protocol): extend droppedFields write-observability to bulk paths + client SDK (#3455) Follow-up to #3448 (#3431 D2). Single-write PATCH/POST /data already surfaces LEGALLY-stripped write fields (readonly #2948 / readonlyWhen #3042 / #3043 create ingress) as `droppedFields`; the bulk paths dropped them silently. This closes out the deferred波及面. - metadata-protocol: updateManyData + batchData collect per-row onFieldsDropped and attach to each result row; insertManyData attaches per-row via ingress diff; createManyData returns an aggregated top-level droppedFields (no per-row slot; static-readonly strip is schema-uniform). Correctness fix: updateManyData and batchData never threaded the caller `context` — bulk writes ran context-less (RLS/FLS/readonlyWhen without the principal; batch create strip forced non-system). All engine calls now run under the resolved context. - spec: BatchOperationResultSchema gains optional per-row droppedFields (covers updateMany + batch); CreateManyDataResponseSchema gains the aggregated one. Both omit-when-empty. No X-ObjectStack-Dropped-Fields header for batches by design — one header cannot express per-row drops, so the body is the canonical channel. - client: CreateDataResult / UpdateDataResult gain droppedFields?: DroppedFieldsEvent[]. - hono adapter + plugin-hono-server: x-objectstack-dropped-fields added to the default Access-Control-Expose-Headers (lockstep across both CORS sites). - Design decision: kept precise droppedFields (reused DroppedFieldsEventSchema) over a generic warnings envelope, for symmetry with the shipped single-write path. - GraphQL item is a no-op: GraphQL has no runtime (kernel.graphql unassigned, handleGraphQL 501s, discovery never advertises it) — nothing to wire until an engine lands, at which point the protocol-layer droppedFields is already present. Tests: 10 bulk protocol cases (per-row + aggregated + context threading + returnRecords=false) and a CORS default-expose assertion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(spec): regenerate api reference for bulk droppedFields fields (#3455) Generated from the batch/protocol Zod schemas — BatchOperationResult (per-row) and CreateManyDataResponse (aggregated) gained droppedFields. Regenerated via gen:schema && gen:docs; these files are generated, not hand-edited. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(spec): regenerate objectstack-api skill references for droppedFields transitive deps (#3455) batch.zod.ts now imports DroppedFieldsEventSchema from data-engine.zod, whose transitive imports (kernel/execution-context, security/explain) pull those into the objectstack-api skill's referenced-schema set. Regenerated via gen:skill-refs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent c88eeda commit c2d9098

13 files changed

Lines changed: 436 additions & 32 deletions

File tree

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
---
2+
"@objectstack/spec": minor
3+
"@objectstack/metadata-protocol": minor
4+
"@objectstack/client": minor
5+
"@objectstack/hono": patch
6+
"@objectstack/plugin-hono-server": patch
7+
---
8+
9+
feat(rest/protocol): extend droppedFields write-observability to the bulk paths + client SDK (#3455)
10+
11+
Follow-up to #3448 (#3431 D2): the single-write PATCH/POST `/data` paths already
12+
surface LEGALLY-stripped write fields (static `readonly` #2948 / `readonlyWhen`
13+
#3042 / #3043 create ingress) as `droppedFields`. The **bulk** write paths did
14+
not — the same strips happened silently on every batched row — and the typed
15+
client warning + CORS mirror were deferred. This closes those out.
16+
17+
**Bulk passthrough (metadata-protocol).**
18+
- `updateManyData` and `batchData` (update/upsert rows) now register a per-row
19+
`onFieldsDropped` collector and attach the events to that row's result.
20+
- `createManyData` diffs each supplied row against its #3043-stripped form and
21+
returns an **aggregated** top-level `droppedFields` (one event per
22+
object/reason with the union of field names) — its `{ records, count }`
23+
response has no per-row slot, and the insert-time strip is static-`readonly`
24+
only, so it is schema-uniform across rows and the aggregate is faithful.
25+
- `insertManyData` keeps per-row precision, attaching `droppedFields` to each
26+
outcome.
27+
- **Correctness fix bundled in:** `updateManyData` and `batchData` never threaded
28+
the caller's execution `context` to the engine — bulk writes ran context-less,
29+
so RLS/FLS and `readonlyWhen` evaluated without the caller's principal, and the
30+
batch create-ingress strip was hard-coded to a non-system context. All engine
31+
calls in both methods now run under the resolved `context`.
32+
33+
**Contract (spec).** `BatchOperationResultSchema` gains an optional per-row
34+
`droppedFields` (covers `updateMany` + `batch`, which alias
35+
`BatchUpdateResponseSchema`); `CreateManyDataResponseSchema` gains the optional
36+
aggregated `droppedFields`. Both are omit-when-empty, so existing clients are
37+
unaffected. `X-ObjectStack-Dropped-Fields` is deliberately **not** emitted for
38+
batches — one response header cannot express per-row drops, so the per-row body
39+
field is the canonical bulk channel.
40+
41+
**Typed client warnings (@objectstack/client).** `CreateDataResult` /
42+
`UpdateDataResult` gain `droppedFields?: DroppedFieldsEvent[]`, giving the body
43+
channel a type instead of an untyped property.
44+
45+
**CORS (@objectstack/hono, @objectstack/plugin-hono-server).**
46+
`x-objectstack-dropped-fields` is added to the default `Access-Control-Expose-Headers`
47+
allow-list (kept in lockstep across both Hono CORS sites) so a cross-origin
48+
browser can read the single-write drop header. The body `droppedFields` remains
49+
the primary, cross-origin-safe surface — this is a convenience mirror.
50+
51+
**GraphQL — not applicable (documented).** #3455 lists a GraphQL mutation item,
52+
but GraphQL has no runtime: `kernel.graphql` is unassigned everywhere and
53+
`handleGraphQL` returns `501`, and discovery never advertises `/graphql`. There
54+
is no schema generator or mutation resolver to expose a typed payload field on,
55+
so there is nothing to wire until a GraphQL engine lands — at which point the
56+
protocol-layer `droppedFields` is already present and only the GraphQL schema
57+
projection would remain.

content/docs/references/api/batch.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ const result = BatchConfig.parse(data);
6363
| **errors** | `{ code: string; message: string; category?: string; details?: any; … }[]` | optional | Array of errors if operation failed |
6464
| **data** | `Record<string, any>` | optional | Full record data (if returnRecords=true) |
6565
| **index** | `number` | optional | Index of the record in the request array |
66+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when'> }[]` | optional | Write-observability (#3407/#3431/#3455): caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` (#2948) / TRUE `readonlyWhen` (#3042) on update, or the #3043 create-ingress strip. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
6667

6768

6869
---

content/docs/references/api/protocol.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,7 @@ const result = AiInsightsRequest.parse(data);
245245
| **object** | `string` || Object name |
246246
| **records** | `Record<string, any>[]` || Created records |
247247
| **count** | `number` || Number of records created |
248+
| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when'> }[]` | optional | Write-observability (#3407/#3431/#3455): caller-supplied `readonly` fields the #3043 create-ingress strip removed before the rows were written. AGGREGATED across the batch (one event per object/reason with the union of dropped field names) rather than per-row, because the insert-time strip is static-`readonly` only — schema-uniform, so every row drops the same set. Present ONLY when ≥1 field was dropped; the creates still succeeded without them (count/success unchanged). Optional — omit-when-empty keeps the shape backward-compatible. (The per-row `insertMany`/`batch` paths carry per-row `droppedFields` on each result instead — see BatchOperationResultSchema.) |
248249

249250

250251
---

packages/adapters/hono/src/hono.test.ts

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,6 +1053,19 @@ describe('createHonoApp', () => {
10531053
expect(exposed.toLowerCase()).toContain('set-auth-token');
10541054
});
10551055

1056+
// [#3455] The single-write `X-ObjectStack-Dropped-Fields` warning header must
1057+
// be readable cross-origin; kept in lockstep with plugin-hono-server's default.
1058+
it('always exposes x-objectstack-dropped-fields by default', async () => {
1059+
const app = createHonoApp({ kernel: mockKernel, prefix: '/api/v1' });
1060+
1061+
const res = await app.request('/api/v1/meta', {
1062+
method: 'GET',
1063+
headers: { Origin: 'https://app.example.com' },
1064+
});
1065+
const exposed = (res.headers.get('access-control-expose-headers') || '').toLowerCase();
1066+
expect(exposed).toContain('x-objectstack-dropped-fields');
1067+
});
1068+
10561069
it('merges user-supplied exposeHeaders with set-auth-token (does not replace)', async () => {
10571070
const app = createHonoApp({
10581071
kernel: mockKernel,

packages/adapters/hono/src/index.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -177,7 +177,10 @@ export function createHonoApp(options: ObjectStackHonoOptions): Hono {
177177
//
178178
// This mirrors `plugin-hono-server`'s CORS wiring — all three
179179
// Hono-based CORS sites must stay in lockstep on this default.
180-
const defaultExposeHeaders = ['set-auth-token'];
180+
// `x-objectstack-dropped-fields` (#3455) lets a cross-origin browser read
181+
// the single-write drop header (#3431); the body `droppedFields` channel is
182+
// the primary, cross-origin-safe surface, so this is a convenience mirror.
183+
const defaultExposeHeaders = ['set-auth-token', 'x-objectstack-dropped-fields'];
181184
const exposeHeaders = Array.from(new Set([
182185
...defaultExposeHeaders,
183186
...(corsOpts.exposeHeaders ?? []),

packages/client/src/index.ts

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

3-
import { QueryAST, SortNode, AggregationNode, isFilterAST } from '@objectstack/spec/data';
3+
import { QueryAST, SortNode, AggregationNode, isFilterAST, type DroppedFieldsEvent } from '@objectstack/spec/data';
44
import {
55
BatchUpdateRequest,
66
BatchUpdateResponse,
@@ -218,13 +218,28 @@ export interface CreateDataResult<T = any> {
218218
object: string;
219219
id: string;
220220
record: T;
221+
/**
222+
* [#3431/#3455] Caller-supplied fields the server LEGALLY stripped before the
223+
* record was written — e.g. a non-system create cannot seed a static `readonly`
224+
* column (#3043), so those keys are dropped and the field re-derives its default.
225+
* Present only when ≥1 field was dropped; the create still succeeded. REST also
226+
* mirrors this in the `X-ObjectStack-Dropped-Fields` response header.
227+
*/
228+
droppedFields?: DroppedFieldsEvent[];
221229
}
222230

223231
/** Spec: UpdateDataResponseSchema */
224232
export interface UpdateDataResult<T = any> {
225233
object: string;
226234
id: string;
227235
record: T;
236+
/**
237+
* [#3431/#3455] Caller-supplied fields the server LEGALLY stripped from the
238+
* write before persisting — static `readonly` (#2948) or a TRUE `readonlyWhen`
239+
* predicate (#3042). Present only when ≥1 field was dropped; the update still
240+
* succeeded. REST also mirrors this in the `X-ObjectStack-Dropped-Fields` header.
241+
*/
242+
droppedFields?: DroppedFieldsEvent[];
228243
}
229244

230245
/** Spec: DeleteDataResponseSchema */
Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
//
3+
// [#3455] Extends the single-write drop-observability of #3431 to the BULK
4+
// write paths. Each bulk method must (a) surface the same LEGAL strips
5+
// (static `readonly` #2948 / `readonlyWhen` #3042 / #3043 create ingress) that
6+
// single-write now reports, and (b) thread the caller's execution `context` to
7+
// the engine so RLS/FLS/`readonlyWhen` run under the caller — a gap the
8+
// pre-#3455 `updateManyData`/`batchData` loops had. Channels:
9+
// - updateManyData / batchData → per-row `droppedFields` on each result row;
10+
// - insertManyData → per-row `droppedFields` on each outcome;
11+
// - createManyData → aggregated top-level `droppedFields` (its
12+
// response has no per-row slot; the insert strip is schema-uniform).
13+
14+
import { describe, it, expect, vi } from 'vitest';
15+
import { ObjectStackProtocolImplementation } from './protocol.js';
16+
17+
const SCHEMA = {
18+
name: 'approval_case',
19+
fields: {
20+
title: { name: 'title', type: 'text' },
21+
approval_status: { name: 'approval_status', type: 'text', readonly: true, defaultValue: 'draft' },
22+
},
23+
};
24+
25+
describe('updateManyData — per-row droppedFields + context threading (#3455)', () => {
26+
it('surfaces per-row engine strips and threads the caller context to each update', async () => {
27+
const update = vi.fn(async (object: string, data: any, options?: any) => {
28+
// Only the second row forges the readonly field → only it drops.
29+
if (data.approval_status !== undefined) {
30+
options?.onFieldsDropped?.({ object, fields: ['approval_status'], reason: 'readonly' });
31+
}
32+
return { id: options.where.id, title: data.title };
33+
});
34+
const engine = { registry: { getObject: () => SCHEMA }, update, findOne: vi.fn(async () => null) };
35+
const p = new ObjectStackProtocolImplementation(engine as any);
36+
37+
const ctx = { userId: 'u1' };
38+
const res: any = await p.updateManyData({
39+
object: 'approval_case',
40+
records: [
41+
{ id: 'rec-1', data: { title: 'A' } },
42+
{ id: 'rec-2', data: { title: 'B', approval_status: 'approved' } },
43+
],
44+
context: ctx,
45+
} as any);
46+
47+
// Row 0 dropped nothing → no droppedFields key; row 1 dropped approval_status.
48+
expect(res.results[0]).not.toHaveProperty('droppedFields');
49+
expect(res.results[1].droppedFields).toEqual([
50+
{ object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
51+
]);
52+
// [#3455] The pre-fix loop never threaded context — assert every engine
53+
// call now runs under the caller's principal.
54+
expect(update).toHaveBeenCalledTimes(2);
55+
expect(update.mock.calls[0][2].context).toBe(ctx);
56+
expect(update.mock.calls[1][2].context).toBe(ctx);
57+
expect(res.succeeded).toBe(2);
58+
});
59+
});
60+
61+
describe('createManyData — aggregated top-level droppedFields (#3455)', () => {
62+
function makeProtocol() {
63+
const engine = {
64+
registry: { getObject: (n: string) => (n === 'approval_case' ? SCHEMA : undefined) },
65+
insert: vi.fn(async (_object: string, rows: any[]) => rows.map((r, i) => ({ id: `rec-${i + 1}`, ...r }))),
66+
};
67+
return { p: new ObjectStackProtocolImplementation(engine as any), engine };
68+
}
69+
70+
it('aggregates the schema-uniform ingress strip across rows into one event', async () => {
71+
const { p } = makeProtocol();
72+
const res: any = await p.createManyData({
73+
object: 'approval_case',
74+
records: [
75+
{ title: 'A', approval_status: 'approved' },
76+
{ title: 'B', approval_status: 'approved' },
77+
],
78+
context: { userId: 'u1' },
79+
});
80+
// Union, not one-event-per-row: both rows dropped the same readonly field.
81+
expect(res.droppedFields).toEqual([
82+
{ object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
83+
]);
84+
expect(res.count).toBe(2);
85+
expect(res.records[0]).not.toHaveProperty('approval_status');
86+
});
87+
88+
it('omits droppedFields when no row seeds a readonly field', async () => {
89+
const { p } = makeProtocol();
90+
const res: any = await p.createManyData({
91+
object: 'approval_case',
92+
records: [{ title: 'A' }, { title: 'B' }],
93+
context: { userId: 'u1' },
94+
});
95+
expect(res).not.toHaveProperty('droppedFields');
96+
});
97+
98+
it('a system-context bulk create keeps the field — no strip, no droppedFields', async () => {
99+
const { p } = makeProtocol();
100+
const res: any = await p.createManyData({
101+
object: 'approval_case',
102+
records: [{ title: 'A', approval_status: 'approved' }],
103+
context: { isSystem: true },
104+
});
105+
expect(res).not.toHaveProperty('droppedFields');
106+
expect(res.records[0].approval_status).toBe('approved');
107+
});
108+
});
109+
110+
describe('insertManyData — per-row droppedFields on outcomes (#3455)', () => {
111+
it('attaches the ingress strip to the matching outcome row only', async () => {
112+
const insertMany = vi.fn(async (_object: string, rows: any[]) =>
113+
rows.map((r, i) => ({ ok: true, record: { id: `rec-${i + 1}`, ...r } })),
114+
);
115+
const engine = { registry: { getObject: () => SCHEMA }, insertMany };
116+
const p = new ObjectStackProtocolImplementation(engine as any);
117+
118+
const res: any = await p.insertManyData({
119+
object: 'approval_case',
120+
records: [
121+
{ title: 'A' },
122+
{ title: 'B', approval_status: 'approved' },
123+
],
124+
context: { userId: 'u1' },
125+
});
126+
127+
expect(res.outcomes[0]).not.toHaveProperty('droppedFields');
128+
expect(res.outcomes[1].droppedFields).toEqual([
129+
{ object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
130+
]);
131+
// The strip really removed the field from what the engine inserted.
132+
expect(insertMany.mock.calls[0][1][1]).not.toHaveProperty('approval_status');
133+
});
134+
});
135+
136+
describe('batchData — per-row droppedFields + context threading (#3455)', () => {
137+
it('create rows surface the ingress strip and honour a system context', async () => {
138+
const insert = vi.fn(async (_object: string, data: any, _options?: any) => ({ id: 'rec-1', ...data }));
139+
const engine = { registry: { getObject: () => SCHEMA }, insert, update: vi.fn(), findOne: vi.fn() };
140+
const p = new ObjectStackProtocolImplementation(engine as any);
141+
142+
const res: any = await p.batchData({
143+
object: 'approval_case',
144+
request: {
145+
operation: 'create',
146+
records: [{ data: { title: 'A', approval_status: 'approved' } }],
147+
},
148+
context: { userId: 'u1' },
149+
} as any);
150+
151+
expect(res.results[0].droppedFields).toEqual([
152+
{ object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
153+
]);
154+
expect(res.results[0].record).not.toHaveProperty('approval_status');
155+
// [#3455] context is threaded to the insert (was hard-coded undefined before).
156+
expect(insert.mock.calls[0][2].context).toEqual({ userId: 'u1' });
157+
});
158+
159+
it('a system-context batch create is exempt from the strip (context now threaded to it)', async () => {
160+
const insert = vi.fn(async (_object: string, data: any) => ({ id: 'rec-1', ...data }));
161+
const engine = { registry: { getObject: () => SCHEMA }, insert, update: vi.fn(), findOne: vi.fn() };
162+
const p = new ObjectStackProtocolImplementation(engine as any);
163+
164+
const res: any = await p.batchData({
165+
object: 'approval_case',
166+
request: { operation: 'create', records: [{ data: { title: 'A', approval_status: 'approved' } }] },
167+
context: { isSystem: true },
168+
} as any);
169+
170+
expect(res.results[0]).not.toHaveProperty('droppedFields');
171+
expect(res.results[0].record.approval_status).toBe('approved');
172+
});
173+
174+
it('update rows surface the engine strip and keep droppedFields when returnRecords=false', async () => {
175+
const update = vi.fn(async (object: string, _data: any, options?: any) => {
176+
options?.onFieldsDropped?.({ object, fields: ['approval_status'], reason: 'readonly' });
177+
return { id: options.where.id };
178+
});
179+
const engine = { registry: { getObject: () => SCHEMA }, update, insert: vi.fn(), findOne: vi.fn() };
180+
const p = new ObjectStackProtocolImplementation(engine as any);
181+
182+
const res: any = await p.batchData({
183+
object: 'approval_case',
184+
request: {
185+
operation: 'update',
186+
records: [{ id: 'rec-1', data: { approval_status: 'approved' } }],
187+
options: { returnRecords: false },
188+
},
189+
context: { userId: 'u1' },
190+
} as any);
191+
192+
// returnRecords:false drops `record` but MUST keep the warning.
193+
expect(res.results[0]).not.toHaveProperty('record');
194+
expect(res.results[0].droppedFields).toEqual([
195+
{ object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
196+
]);
197+
expect(update.mock.calls[0][2].context).toEqual({ userId: 'u1' });
198+
});
199+
});

0 commit comments

Comments
 (0)