feat(objectql)!: media value shapes enforce per deployment, once its file migration verified (#3438)

#3438 scheduled the strict flip for "a later minor once telemetry is quiet" —
our telemetry, deciding for their deployments. #3617 built the mechanism that
replaces that: a deployment's own migration flag. This lands the half that flag
actually covers.

A malformed media value now rejects with `invalid_type` instead of warning, but
only where `os migrate files-to-references --apply` has run and passed its
self-check. The legacy forms this rejects — inline `{url, name, …}` blobs, bare
URLs — are exactly what that migration converts, so a verified deployment has
been shown to hold none, while an unmigrated one is untouched rather than having
every media update start failing on upgrade.

## Only media, deliberately

#3438 packaged three things as one flip, and the flag covers one of them. It
asserts "this deployment's FILE values are migrated and reconciled". It says
nothing about whether a `lookup` id or a `location` payload is well formed, and
nothing at all about D2's action parameters. Gating those on it would be
borrowing evidence for a fact it does not cover — an authority answering a
question it was not asked, which is the same shape as the read-resolver defect
#3617 fixed one layer down. Reference and structured-JSON classes keep their
own warn-first rollout until something can vouch for them.

`OS_DATA_VALUE_SHAPE_STRICT_ENABLED` is unchanged (still opts every class in,
still forces media strict on an unmigrated deployment). New escape hatch
`OS_ALLOW_LAX_MEDIA_VALUES=1` re-opens media on a verified deployment; when both
are set the lenient one wins, because a warning nobody reads costs less than an
app that stops writing.

## Cost

The fact lives in the database while the validator is synchronous and per-write,
so the engine reads it once and memoizes — and stays dormant unless the written
object declares a media field, mirroring the rule the storage module already
states. One query per process for apps that store files, zero for those that
do not. Every way of not knowing (no storage service, no row, unreadable table,
malformed row) answers "not verified": a deployment whose evidence cannot be
read keeps writing rather than starting to reject.

Verified end-to-end on showcase + SQLite against one database: the same write
was accepted before the migration, rejected with `invalid_type` after it, and
accepted again under OS_ALLOW_LAX_MEDIA_VALUES.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V9uWWyKq6pNPQthwCXL8ma

Author: claude

.changeset/adr-0104-d1-media-strict-per-deployment.md

@@ -0,0 +1,42 @@
+---
+"@objectstack/objectql": minor
+---
+
+feat(objectql)!: media value shapes enforce once THIS deployment has verified its file migration (#3438 D1 media half, gated by #3617)
+
+A `file` / `image` / `avatar` / `video` / `audio` value that does not match the
+stored contract (an opaque `sys_file` id) now **rejects with `invalid_type`**
+instead of warning — but only on a deployment that has run
+`os migrate files-to-references --apply` and passed its self-check.
+
+**Why this is not a version-wide flip.** The legacy media values this rejects —
+inline `{url, name, …}` blobs, bare URLs — are exactly what that migration
+converts. A deployment that has run it has been *shown* to hold none; a
+deployment that has not would have every media-field update start failing the
+moment it upgraded. So the enforcement follows the evidence, per deployment,
+rather than the release. Nothing changes for a deployment until it migrates.
+
+**Upgrading:**
+
+```bash
+os migrate files-to-references          # dry run: reports what would convert
+os migrate files-to-references --apply  # convert, verify, record the flag
+```
+
+If a write starts failing after you migrate, the value genuinely does not match
+the contract — the error names the field. `OS_ALLOW_LAX_MEDIA_VALUES=1` re-opens
+media leniency while you diagnose.
+
+**Scope — deliberately only media.** `OS_DATA_VALUE_SHAPE_STRICT_ENABLED` is
+unchanged and still opts every class into strict (and still forces media strict
+on a deployment that has not migrated). Reference types (`lookup`, `user`, …)
+and structured JSON (`location`, `address`, `repeater`, …) stay warn-first: the
+file migration is evidence about file values and says nothing about whether a
+`location` is well formed, so gating them on its flag would be borrowing
+evidence for a fact it does not cover. They flip when something can vouch for
+them — see #3438.
+
+**Cost.** Dormant unless the written object declares a media field, and the
+flag read is memoized, so this is one query per process for apps that store
+files and zero for those that do not. A running server picks up a
+newly-recorded migration on restart, or via `engine.invalidateDataMigrationFlags()`.

content/docs/deployment/cli.mdx

@@ -545,18 +545,31 @@ field instead.
 
 The run then reconciles what records actually hold against what `sys_file`
 records as each file's owner. Zero blocking discrepancies is what records the
-flag — and that flag, not the version number, is what later enables features
-that act irreversibly on migrated data (reclaiming the bytes of released files;
-rejecting rather than warning about legacy media values). Never running it is
-safe: files are simply retained forever.
+flag — and that flag, not the version number, is what enables behaviour that
+depends on the data actually being migrated. Never running it is safe: files
+are simply retained forever, and media values keep warning instead of failing.
 
 Exit status is `0` only when the self-check passes, so CI can gate on it.
 
+#### What the flag turns on
+
+| Once verified | Effect |
+| :--- | :--- |
+| **Media value shapes** | A malformed `file` / `image` / `avatar` / `video` / `audio` value is **rejected** (`400 invalid_type`) instead of warned about. Set `OS_ALLOW_LAX_MEDIA_VALUES=1` to re-open leniency while diagnosing. |
+| **Released-file collection** | Not yet shipped — the flag is the gate it will read. |
+
+Other value classes are unaffected: a `lookup` or `location` value keeps its own
+warn-first rollout, because this migration is evidence about *file* values and
+says nothing about theirs.
+
 <Callout type="warn">
 A dry run writes **nothing** — not the conversions, and not the flag either,
 even when the self-check would pass. `--apply` is the only writing mode. A
 later run that *fails* its self-check clears the flag's verified state, so a
 database that has drifted closes its own gate.
+
+A running server reads the flag once; after migrating, **restart it** for
+enforcement to take effect.
 </Callout>
 
 ### Scaffolding

docs/adr/0104-field-runtime-value-shape-contract.md

@@ -419,8 +419,12 @@ Therefore the target end-state is **two enforcement points, each with one job**:
 - **Build/validate time → hard reject.** Net-new metadata (the AI's output)
   must fail loudly at authoring. This is the primary defence against AI error.
 - **Runtime → warn-first, then flip.** Deployed data keeps working through the
-  warn window; the flip to strict-by-default (tracked in #3438) closes it once
-  telemetry is quiet.
+  warn window; the flip to strict-by-default (tracked in #3438) closes it.
+  *(Amended by the second 2026-07-27 addendum: "once telemetry is quiet" was
+  our telemetry deciding for their deployments. The **media** half now flips
+  per deployment, when that deployment's own file-as-reference migration has
+  verified. The reference and structured-JSON halves still await an evidence
+  source of their own — the file migration does not vouch for them.)*
 
 This refines D2 (today runtime-only) and reshapes the #3438 flip: the priority
 is adding the **build-time** rejection for value shapes and action params, not
@@ -670,6 +674,28 @@ a deployment when that deployment has completed and verified its migration —
 not when a version number arrives. One flag, not two gates that can disagree:
 they are gating on the same fact.
 
+**Amendment while implementing this (2026-07-27, later still).** "#3438 reads
+the flag" was too broad: #3438 is three things, and the flag only covers one.
+The flag asserts *this deployment's file-field values have been migrated and
+reconciled*. That is evidence about **media** value shapes and nothing else —
+it says nothing about whether a `lookup` id or a `location` payload is well
+formed, and nothing at all about D2's action parameters. Gating those on it
+would be borrowing evidence for a fact it does not cover, which is the same
+error one layer down: an authority answering a question it was not asked.
+
+So the split is:
+
+| | evidence | status |
+|---|---|---|
+| D1 media (`file`/`image`/`avatar`/`video`/`audio`) | the `adr-0104-file-references` flag | flips per deployment |
+| D1 references + structured JSON | none yet — needs its own | stays warn-first |
+| D2 action params | unrelated to any data migration | stays warn-first |
+
+The last two are not blocked on the mechanism — it exists and works. They are
+blocked on someone deciding what would constitute evidence that a deployment's
+`location` values are sound, which is a different question from the one this
+addendum answers.
+
 ### What cannot be automated, and does not block
 
 Whether an **external URL** (`https://cdn…`) should become an explicit `url`

packages/cli/src/commands/migrate/files-to-references.ts

@@ -253,7 +253,9 @@ export default class MigrateFilesToReferences extends Command {
         if (apply) {
           printSuccess(
             'Self-check passed — deployment flag recorded (adr-0104-file-references). ' +
-              'This deployment is now eligible for strict media enforcement and, once shipped, file collection.',
+              'Media value shapes are now ENFORCED on this deployment: a malformed ' +
+              'file/image value is rejected rather than warned about. ' +
+              '(Set OS_ALLOW_LAX_MEDIA_VALUES=1 to re-open leniency while diagnosing.)',
           );
         } else if (result.backfill.converted > 0) {
           printInfo(

packages/objectql/src/engine.test.ts

@@ -2075,3 +2075,132 @@ describe('ObjectQL Engine', () => {
         });
     });
 });
+
+/**
+ * #3617 / #3438 — the engine carries the deployment's own migration evidence
+ * into the (synchronous, per-write) value-shape validator.
+ *
+ * The fact lives in `sys_migration`, so it needs a database read; the write
+ * path needs it on every insert/update. These prove the read happens once,
+ * and that every way of NOT knowing lands on lenient — a deployment whose
+ * evidence cannot be read keeps writing rather than starting to reject.
+ */
+describe('ObjectQL — file-as-reference migration flag (#3617)', () => {
+  let engine: ObjectQL;
+  let driver: IDataDriver;
+
+  const verifiedRow = {
+    id: 'adr-0104-file-references',
+    last_run_at: '2026-07-27T00:00:00.000Z',
+    verified_at: '2026-07-27T00:00:00.000Z',
+    blocking: 0,
+  };
+
+  beforeEach(() => {
+    vi.clearAllMocks();
+    driver = {
+      name: 'default-driver',
+      connect: vi.fn().mockResolvedValue(undefined),
+      disconnect: vi.fn().mockResolvedValue(undefined),
+      find: vi.fn().mockResolvedValue([]),
+      findOne: vi.fn(),
+      create: vi.fn().mockResolvedValue({ id: '1' }),
+      update: vi.fn().mockResolvedValue({ id: '1' }),
+      delete: vi.fn(),
+      count: vi.fn(),
+      capabilities: {} as any,
+    } as unknown as IDataDriver;
+    engine = new ObjectQL();
+    engine.registerDriver(driver, true);
+  });
+
+  /** `doc` holds a legacy inline blob — exactly what the migration converts. */
+  const withMediaObject = () => {
+    vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) => {
+      if (name === 'note') return { name: 'note', fields: { doc: { type: 'file' } } } as any;
+      if (name === 'sys_migration') return { name: 'sys_migration', fields: { id: { type: 'text' } } } as any;
+      return undefined;
+    });
+  };
+  const legacyBlob = { doc: { url: 'https://cdn/f.pdf', name: 'f.pdf' } };
+
+  it('a verified flag makes a malformed media value reject', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockResolvedValue([verifiedRow] as any);
+
+    await expect(engine.insert('note', { ...legacyBlob })).rejects.toThrow(/invalid file value/i);
+  });
+
+  it('no flag row → lenient (the write goes through)', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockResolvedValue([]);
+
+    await expect(engine.insert('note', { ...legacyBlob })).resolves.toBeDefined();
+  });
+
+  it('a failed run (verified_at cleared) → lenient', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockResolvedValue([{ ...verifiedRow, verified_at: null, blocking: 3 }] as any);
+
+    await expect(engine.insert('note', { ...legacyBlob })).resolves.toBeDefined();
+  });
+
+  it('an unreadable sys_migration → lenient, and the write is not failed by the probe', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockRejectedValue(new Error('no such table: sys_migration'));
+
+    await expect(engine.insert('note', { ...legacyBlob })).resolves.toBeDefined();
+  });
+
+  /**
+   * Dormancy, mirroring the storage module's own rule: an object with no
+   * file-class field can hold no malformed media value, so it must not pay
+   * even one query to learn that. Nearly every object is in this case.
+   */
+  it('costs no query for an object that declares no media field', async () => {
+    vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) => {
+      if (name === 'invoice') return { name: 'invoice', fields: { amount: { type: 'number' } } } as any;
+      if (name === 'sys_migration') return { name: 'sys_migration', fields: { id: { type: 'text' } } } as any;
+      return undefined;
+    });
+    vi.mocked(driver.find).mockResolvedValue([verifiedRow] as any);
+
+    await expect(engine.insert('invoice', { amount: 10 })).resolves.toBeDefined();
+    expect(vi.mocked(driver.find).mock.calls.filter((c) => c[0] === 'sys_migration')).toHaveLength(0);
+  });
+
+  /** No storage service → no sys_migration object → not even a query. */
+  it('costs no query on a kernel without the storage objects', async () => {
+    vi.mocked(SchemaRegistry.getObject).mockImplementation((name: string) =>
+      name === 'note' ? ({ name: 'note', fields: { doc: { type: 'file' } } } as any) : undefined,
+    );
+    vi.mocked(driver.find).mockResolvedValue([]);
+
+    await expect(engine.insert('note', { ...legacyBlob })).resolves.toBeDefined();
+    expect(driver.find).not.toHaveBeenCalled();
+  });
+
+  it('reads the flag once per process, not once per write', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockResolvedValue([verifiedRow] as any);
+
+    await expect(engine.insert('note', { doc: 'file_01' })).resolves.toBeDefined();
+    await expect(engine.insert('note', { doc: 'file_02' })).resolves.toBeDefined();
+    await expect(engine.insert('note', { doc: 'file_03' })).resolves.toBeDefined();
+
+    const flagReads = vi.mocked(driver.find).mock.calls.filter((c) => c[0] === 'sys_migration');
+    expect(flagReads).toHaveLength(1);
+  });
+
+  it('invalidateDataMigrationFlags forces a re-read', async () => {
+    withMediaObject();
+    vi.mocked(driver.find).mockResolvedValue([verifiedRow] as any);
+
+    await engine.insert('note', { doc: 'file_01' });
+    engine.invalidateDataMigrationFlags();
+    await engine.insert('note', { doc: 'file_02' });
+
+    const flagReads = vi.mocked(driver.find).mock.calls.filter((c) => c[0] === 'sys_migration');
+    expect(flagReads).toHaveLength(2);
+  });
+});