feat(rest/protocol): extend droppedFields write-observability to bulk paths + client SDK (#3455)

.changeset/dropped-fields-bulk-graphql-client.md

@@ -0,0 +1,57 @@
+---
+"@objectstack/spec": minor
+"@objectstack/metadata-protocol": minor
+"@objectstack/client": minor
+"@objectstack/hono": patch
+"@objectstack/plugin-hono-server": patch
+---
+
+feat(rest/protocol): extend droppedFields write-observability to the bulk paths + client SDK (#3455)
+
+Follow-up to #3448 (#3431 D2): the single-write PATCH/POST `/data` paths already
+surface LEGALLY-stripped write fields (static `readonly` #2948 / `readonlyWhen`
+#3042 / #3043 create ingress) as `droppedFields`. The **bulk** write paths did
+not — the same strips happened silently on every batched row — and the typed
+client warning + CORS mirror were deferred. This closes those out.
+
+**Bulk passthrough (metadata-protocol).**
+- `updateManyData` and `batchData` (update/upsert rows) now register a per-row
+  `onFieldsDropped` collector and attach the events to that row's result.
+- `createManyData` diffs each supplied row against its #3043-stripped form and
+  returns an **aggregated** top-level `droppedFields` (one event per
+  object/reason with the union of field names) — its `{ records, count }`
+  response has no per-row slot, and the insert-time strip is static-`readonly`
+  only, so it is schema-uniform across rows and the aggregate is faithful.
+- `insertManyData` keeps per-row precision, attaching `droppedFields` to each
+  outcome.
+- **Correctness fix bundled in:** `updateManyData` and `batchData` never threaded
+  the caller's execution `context` to the engine — bulk writes ran context-less,
+  so RLS/FLS and `readonlyWhen` evaluated without the caller's principal, and the
+  batch create-ingress strip was hard-coded to a non-system context. All engine
+  calls in both methods now run under the resolved `context`.
+
+**Contract (spec).** `BatchOperationResultSchema` gains an optional per-row
+`droppedFields` (covers `updateMany` + `batch`, which alias
+`BatchUpdateResponseSchema`); `CreateManyDataResponseSchema` gains the optional
+aggregated `droppedFields`. Both are omit-when-empty, so existing clients are
+unaffected. `X-ObjectStack-Dropped-Fields` is deliberately **not** emitted for
+batches — one response header cannot express per-row drops, so the per-row body
+field is the canonical bulk channel.
+
+**Typed client warnings (@objectstack/client).** `CreateDataResult` /
+`UpdateDataResult` gain `droppedFields?: DroppedFieldsEvent[]`, giving the body
+channel a type instead of an untyped property.
+
+**CORS (@objectstack/hono, @objectstack/plugin-hono-server).**
+`x-objectstack-dropped-fields` is added to the default `Access-Control-Expose-Headers`
+allow-list (kept in lockstep across both Hono CORS sites) so a cross-origin
+browser can read the single-write drop header. The body `droppedFields` remains
+the primary, cross-origin-safe surface — this is a convenience mirror.
+
+**GraphQL — not applicable (documented).** #3455 lists a GraphQL mutation item,
+but GraphQL has no runtime: `kernel.graphql` is unassigned everywhere and
+`handleGraphQL` returns `501`, and discovery never advertises `/graphql`. There
+is no schema generator or mutation resolver to expose a typed payload field on,
+so there is nothing to wire until a GraphQL engine lands — at which point the
+protocol-layer `droppedFields` is already present and only the GraphQL schema
+projection would remain.

content/docs/references/api/batch.mdx

@@ -63,6 +63,7 @@ const result = BatchConfig.parse(data);
 | **errors** | `{ code: string; message: string; category?: string; details?: any; … }[]` | optional | Array of errors if operation failed |
 | **data** | `Record<string, any>` | optional | Full record data (if returnRecords=true) |
 | **index** | `number` | optional | Index of the record in the request array |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when'> }[]` | optional | Write-observability (#3407/#3431/#3455): caller-supplied fields LEGALLY stripped from THIS row before it was written — static `readonly` (#2948) / TRUE `readonlyWhen` (#3042) on update, or the #3043 create-ingress strip. Per-row because a batch can drop different fields on different rows (`readonlyWhen` is record-state-dependent). Present ONLY when ≥1 field was dropped for this row; the row still succeeded (success unchanged). A single response header cannot express per-row drops, so this body field is the canonical bulk channel — REST does not emit `X-ObjectStack-Dropped-Fields` for batches. Optional — omit-when-empty keeps the shape backward-compatible. |
 
 
 ---

content/docs/references/api/protocol.mdx

@@ -245,6 +245,7 @@ const result = AiInsightsRequest.parse(data);
 | **object** | `string` | ✅ | Object name |
 | **records** | `Record<string, any>[]` | ✅ | Created records |
 | **count** | `number` | ✅ | Number of records created |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when'> }[]` | optional | Write-observability (#3407/#3431/#3455): caller-supplied `readonly` fields the #3043 create-ingress strip removed before the rows were written. AGGREGATED across the batch (one event per object/reason with the union of dropped field names) rather than per-row, because the insert-time strip is static-`readonly` only — schema-uniform, so every row drops the same set. Present ONLY when ≥1 field was dropped; the creates still succeeded without them (count/success unchanged). Optional — omit-when-empty keeps the shape backward-compatible. (The per-row `insertMany`/`batch` paths carry per-row `droppedFields` on each result instead — see BatchOperationResultSchema.) |
 
 
 ---

packages/adapters/hono/src/hono.test.ts

@@ -1053,6 +1053,19 @@ describe('createHonoApp', () => {
       expect(exposed.toLowerCase()).toContain('set-auth-token');
     });
 
+    // [#3455] The single-write `X-ObjectStack-Dropped-Fields` warning header must
+    // be readable cross-origin; kept in lockstep with plugin-hono-server's default.
+    it('always exposes x-objectstack-dropped-fields by default', async () => {
+      const app = createHonoApp({ kernel: mockKernel, prefix: '/api/v1' });
+
+      const res = await app.request('/api/v1/meta', {
+        method: 'GET',
+        headers: { Origin: 'https://app.example.com' },
+      });
+      const exposed = (res.headers.get('access-control-expose-headers') || '').toLowerCase();
+      expect(exposed).toContain('x-objectstack-dropped-fields');
+    });
+
     it('merges user-supplied exposeHeaders with set-auth-token (does not replace)', async () => {
       const app = createHonoApp({
         kernel: mockKernel,

packages/adapters/hono/src/index.ts

@@ -177,7 +177,10 @@ export function createHonoApp(options: ObjectStackHonoOptions): Hono {
       //
       // This mirrors `plugin-hono-server`'s CORS wiring — all three
       // Hono-based CORS sites must stay in lockstep on this default.
-      const defaultExposeHeaders = ['set-auth-token'];
+      // `x-objectstack-dropped-fields` (#3455) lets a cross-origin browser read
+      // the single-write drop header (#3431); the body `droppedFields` channel is
+      // the primary, cross-origin-safe surface, so this is a convenience mirror.
+      const defaultExposeHeaders = ['set-auth-token', 'x-objectstack-dropped-fields'];
       const exposeHeaders = Array.from(new Set([
         ...defaultExposeHeaders,
         ...(corsOpts.exposeHeaders ?? []),

packages/client/src/index.ts

@@ -1,6 +1,6 @@
 // Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
 
-import { QueryAST, SortNode, AggregationNode, isFilterAST } from '@objectstack/spec/data';
+import { QueryAST, SortNode, AggregationNode, isFilterAST, type DroppedFieldsEvent } from '@objectstack/spec/data';
 import {
   BatchUpdateRequest,
   BatchUpdateResponse,
@@ -218,13 +218,28 @@ export interface CreateDataResult<T = any> {
   object: string;
   id: string;
   record: T;
+  /**
+   * [#3431/#3455] Caller-supplied fields the server LEGALLY stripped before the
+   * record was written — e.g. a non-system create cannot seed a static `readonly`
+   * column (#3043), so those keys are dropped and the field re-derives its default.
+   * Present only when ≥1 field was dropped; the create still succeeded. REST also
+   * mirrors this in the `X-ObjectStack-Dropped-Fields` response header.
+   */
+  droppedFields?: DroppedFieldsEvent[];
 }
 
 /** Spec: UpdateDataResponseSchema */
 export interface UpdateDataResult<T = any> {
   object: string;
   id: string;
   record: T;
+  /**
+   * [#3431/#3455] Caller-supplied fields the server LEGALLY stripped from the
+   * write before persisting — static `readonly` (#2948) or a TRUE `readonlyWhen`
+   * predicate (#3042). Present only when ≥1 field was dropped; the update still
+   * succeeded. REST also mirrors this in the `X-ObjectStack-Dropped-Fields` header.
+   */
+  droppedFields?: DroppedFieldsEvent[];
 }
 
 /** Spec: DeleteDataResponseSchema */

packages/metadata-protocol/src/protocol.dropped-fields.bulk.test.ts

@@ -0,0 +1,199 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+//
+// [#3455] Extends the single-write drop-observability of #3431 to the BULK
+// write paths. Each bulk method must (a) surface the same LEGAL strips
+// (static `readonly` #2948 / `readonlyWhen` #3042 / #3043 create ingress) that
+// single-write now reports, and (b) thread the caller's execution `context` to
+// the engine so RLS/FLS/`readonlyWhen` run under the caller — a gap the
+// pre-#3455 `updateManyData`/`batchData` loops had. Channels:
+//   - updateManyData / batchData → per-row `droppedFields` on each result row;
+//   - insertManyData            → per-row `droppedFields` on each outcome;
+//   - createManyData            → aggregated top-level `droppedFields` (its
+//     response has no per-row slot; the insert strip is schema-uniform).
+
+import { describe, it, expect, vi } from 'vitest';
+import { ObjectStackProtocolImplementation } from './protocol.js';
+
+const SCHEMA = {
+  name: 'approval_case',
+  fields: {
+    title: { name: 'title', type: 'text' },
+    approval_status: { name: 'approval_status', type: 'text', readonly: true, defaultValue: 'draft' },
+  },
+};
+
+describe('updateManyData — per-row droppedFields + context threading (#3455)', () => {
+  it('surfaces per-row engine strips and threads the caller context to each update', async () => {
+    const update = vi.fn(async (object: string, data: any, options?: any) => {
+      // Only the second row forges the readonly field → only it drops.
+      if (data.approval_status !== undefined) {
+        options?.onFieldsDropped?.({ object, fields: ['approval_status'], reason: 'readonly' });
+      }
+      return { id: options.where.id, title: data.title };
+    });
+    const engine = { registry: { getObject: () => SCHEMA }, update, findOne: vi.fn(async () => null) };
+    const p = new ObjectStackProtocolImplementation(engine as any);
+
+    const ctx = { userId: 'u1' };
+    const res: any = await p.updateManyData({
+      object: 'approval_case',
+      records: [
+        { id: 'rec-1', data: { title: 'A' } },
+        { id: 'rec-2', data: { title: 'B', approval_status: 'approved' } },
+      ],
+      context: ctx,
+    } as any);
+
+    // Row 0 dropped nothing → no droppedFields key; row 1 dropped approval_status.
+    expect(res.results[0]).not.toHaveProperty('droppedFields');
+    expect(res.results[1].droppedFields).toEqual([
+      { object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
+    ]);
+    // [#3455] The pre-fix loop never threaded context — assert every engine
+    // call now runs under the caller's principal.
+    expect(update).toHaveBeenCalledTimes(2);
+    expect(update.mock.calls[0][2].context).toBe(ctx);
+    expect(update.mock.calls[1][2].context).toBe(ctx);
+    expect(res.succeeded).toBe(2);
+  });
+});
+
+describe('createManyData — aggregated top-level droppedFields (#3455)', () => {
+  function makeProtocol() {
+    const engine = {
+      registry: { getObject: (n: string) => (n === 'approval_case' ? SCHEMA : undefined) },
+      insert: vi.fn(async (_object: string, rows: any[]) => rows.map((r, i) => ({ id: `rec-${i + 1}`, ...r }))),
+    };
+    return { p: new ObjectStackProtocolImplementation(engine as any), engine };
+  }
+
+  it('aggregates the schema-uniform ingress strip across rows into one event', async () => {
+    const { p } = makeProtocol();
+    const res: any = await p.createManyData({
+      object: 'approval_case',
+      records: [
+        { title: 'A', approval_status: 'approved' },
+        { title: 'B', approval_status: 'approved' },
+      ],
+      context: { userId: 'u1' },
+    });
+    // Union, not one-event-per-row: both rows dropped the same readonly field.
+    expect(res.droppedFields).toEqual([
+      { object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
+    ]);
+    expect(res.count).toBe(2);
+    expect(res.records[0]).not.toHaveProperty('approval_status');
+  });
+
+  it('omits droppedFields when no row seeds a readonly field', async () => {
+    const { p } = makeProtocol();
+    const res: any = await p.createManyData({
+      object: 'approval_case',
+      records: [{ title: 'A' }, { title: 'B' }],
+      context: { userId: 'u1' },
+    });
+    expect(res).not.toHaveProperty('droppedFields');
+  });
+
+  it('a system-context bulk create keeps the field — no strip, no droppedFields', async () => {
+    const { p } = makeProtocol();
+    const res: any = await p.createManyData({
+      object: 'approval_case',
+      records: [{ title: 'A', approval_status: 'approved' }],
+      context: { isSystem: true },
+    });
+    expect(res).not.toHaveProperty('droppedFields');
+    expect(res.records[0].approval_status).toBe('approved');
+  });
+});
+
+describe('insertManyData — per-row droppedFields on outcomes (#3455)', () => {
+  it('attaches the ingress strip to the matching outcome row only', async () => {
+    const insertMany = vi.fn(async (_object: string, rows: any[]) =>
+      rows.map((r, i) => ({ ok: true, record: { id: `rec-${i + 1}`, ...r } })),
+    );
+    const engine = { registry: { getObject: () => SCHEMA }, insertMany };
+    const p = new ObjectStackProtocolImplementation(engine as any);
+
+    const res: any = await p.insertManyData({
+      object: 'approval_case',
+      records: [
+        { title: 'A' },
+        { title: 'B', approval_status: 'approved' },
+      ],
+      context: { userId: 'u1' },
+    });
+
+    expect(res.outcomes[0]).not.toHaveProperty('droppedFields');
+    expect(res.outcomes[1].droppedFields).toEqual([
+      { object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
+    ]);
+    // The strip really removed the field from what the engine inserted.
+    expect(insertMany.mock.calls[0][1][1]).not.toHaveProperty('approval_status');
+  });
+});
+
+describe('batchData — per-row droppedFields + context threading (#3455)', () => {
+  it('create rows surface the ingress strip and honour a system context', async () => {
+    const insert = vi.fn(async (_object: string, data: any, _options?: any) => ({ id: 'rec-1', ...data }));
+    const engine = { registry: { getObject: () => SCHEMA }, insert, update: vi.fn(), findOne: vi.fn() };
+    const p = new ObjectStackProtocolImplementation(engine as any);
+
+    const res: any = await p.batchData({
+      object: 'approval_case',
+      request: {
+        operation: 'create',
+        records: [{ data: { title: 'A', approval_status: 'approved' } }],
+      },
+      context: { userId: 'u1' },
+    } as any);
+
+    expect(res.results[0].droppedFields).toEqual([
+      { object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
+    ]);
+    expect(res.results[0].record).not.toHaveProperty('approval_status');
+    // [#3455] context is threaded to the insert (was hard-coded undefined before).
+    expect(insert.mock.calls[0][2].context).toEqual({ userId: 'u1' });
+  });
+
+  it('a system-context batch create is exempt from the strip (context now threaded to it)', async () => {
+    const insert = vi.fn(async (_object: string, data: any) => ({ id: 'rec-1', ...data }));
+    const engine = { registry: { getObject: () => SCHEMA }, insert, update: vi.fn(), findOne: vi.fn() };
+    const p = new ObjectStackProtocolImplementation(engine as any);
+
+    const res: any = await p.batchData({
+      object: 'approval_case',
+      request: { operation: 'create', records: [{ data: { title: 'A', approval_status: 'approved' } }] },
+      context: { isSystem: true },
+    } as any);
+
+    expect(res.results[0]).not.toHaveProperty('droppedFields');
+    expect(res.results[0].record.approval_status).toBe('approved');
+  });
+
+  it('update rows surface the engine strip and keep droppedFields when returnRecords=false', async () => {
+    const update = vi.fn(async (object: string, _data: any, options?: any) => {
+      options?.onFieldsDropped?.({ object, fields: ['approval_status'], reason: 'readonly' });
+      return { id: options.where.id };
+    });
+    const engine = { registry: { getObject: () => SCHEMA }, update, insert: vi.fn(), findOne: vi.fn() };
+    const p = new ObjectStackProtocolImplementation(engine as any);
+
+    const res: any = await p.batchData({
+      object: 'approval_case',
+      request: {
+        operation: 'update',
+        records: [{ id: 'rec-1', data: { approval_status: 'approved' } }],
+        options: { returnRecords: false },
+      },
+      context: { userId: 'u1' },
+    } as any);
+
+    // returnRecords:false drops `record` but MUST keep the warning.
+    expect(res.results[0]).not.toHaveProperty('record');
+    expect(res.results[0].droppedFields).toEqual([
+      { object: 'approval_case', fields: ['approval_status'], reason: 'readonly' },
+    ]);
+    expect(update.mock.calls[0][2].context).toEqual({ userId: 'u1' });
+  });
+});