feat(vnext): authenticate CTE main query sites (#190)

## Summary

- authenticate bounded CTE main-query entrypoints before the relation
query-site recognizer consumes them
- unify source, statement-index, slot, layout, and built-in runtime
provenance so mixed or stale evidence fails closed
- keep the low-level query-site recognizer independent of CTE and
aggregate runtime modules through a one-way orchestration layer
- preserve independent old/new statement contexts when incremental
indexes reuse immutable slot identities
- validate real ready paths in the CTE/query-site benchmarks

## Correctness and trust boundaries

- package-created sources, statement indexes, exact slots, CTE layouts,
and coherent dialect runtimes are authenticated by identity
- each statement index privately owns its exact analysis text, frozen
lexical-profile scalar snapshot, and bounded slot membership
- foreign sources, indexes, slots, profiles, layouts, mixed runtimes,
mutable structural copies, and hostile proxies return unavailable
- incomplete and invalid CTE headers remain fail-closed while valid main
queries work across PostgreSQL, DuckDB, BigQuery, and Dremio

## Verification

- 1,671 tests passed and 1 expected failure remained expected
- changed coverage: 96.99% statements, 95.95% branches, 100% functions,
96.96% lines
- all five TypeScript configurations passed
- oxlint passed with zero warnings or errors
- test-integrity and diff checks passed
- exact tarball package smoke, SSR import, strict CSP, worker placement,
and bundle budgets passed
- Chromium browser suite passed: 4 files / 7 tests
- statement-index, CTE-layout, and query-site benchmarks passed
- exact SHA c096a16 received three
independent adversarial approvals after five review/fix loops

Part of #169.

<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Authenticate CTE main‑query entrypoints and only recognize relation
query sites when the source, statement index, slot, and dialect all
match. Ambiguous evidence fails closed, keeping the core recognizer
independent; part of #169.

- **New Features**
- Authenticated CTE layouts: `analyzeSqlCteLayout(source, index, slot,
dialect)` records provenance and binds main‑query entrypoints to the
exact source/index/slot/dialect;
`resolveAuthenticatedSqlCteEntrypoints(...)` exposes them safely.
- Orchestration: `recognizeSqlRelationQuerySiteWithCteLayout(...)`
validates runtime/source/slot via `isSqlRelationDialectRuntime`,
`isSqlSourceSnapshot`, and `isSqlStatementSlotSnapshot`, then feeds
authenticated entrypoints into the query‑site recognizer; runtimes are
registered with `registerSqlRelationDialectRuntime`.
- Entrypoint‑aware query‑site:
`recognizeSqlRelationQuerySiteWithEntrypoints(...)` matches SELECTs at
the exact depth/offset for nested CTEs.
- Statement index provenance: `buildSqlStatementIndex` snapshots the
lexical profile; `isExactSqlStatementSlotSnapshot(For)` ensures slots
belong to the index and text/profile; `updateSqlStatementIndex` rebuilds
when text or lexical profile changes.

- **Migration**
- Pass the statement index into `analyzeSqlCteLayout` (new signature:
`(source, index, slot, dialect)`).
- Use `recognizeSqlRelationQuerySiteWithCteLayout` when combining CTE
layout with relation query‑site; keep `recognizeSqlRelationQuerySite`
for non‑CTE cases.
- Use built‑in registered runtimes from `relation-dialect` and authentic
sources/slots produced by our builders.

<sup>Written for commit 9ca185f.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/marimo-team/codemirror-sql/pull/190?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

Author: Light2Dark

src/vnext/__tests__/cte-layout.bench.ts

@@ -17,17 +17,19 @@ const dialect = DUCKDB_SQL_RELATION_DIALECT.cteLayout;
 
 function fixture(text: string): {
   readonly source: ReturnType<typeof createIdentitySqlSource>;
+  readonly index: ReturnType<typeof buildSqlStatementIndex>;
   readonly slot: ExactSqlStatementSlot;
 } {
   const source = createIdentitySqlSource(text);
-  const slot = buildSqlStatementIndex(
+  const index = buildSqlStatementIndex(
     source.analysisText,
     dialect.lexicalProfile,
-  ).slots[0];
+  );
+  const slot = index.slots[0];
   if (!slot || slot.boundaryQuality !== "exact") {
     throw new Error("CTE benchmark fixture requires an exact statement");
   }
-  return { slot, source };
+  return { index, slot, source };
 }
 
 const tenKibibytes = 10 * 1_024;
@@ -62,6 +64,7 @@ const bareFrameHeavyText = `SELECT ${Array.from(
 const bareFrameHeavyFixture = fixture(bareFrameHeavyText);
 const depthLayout = analyzeSqlCteLayout(
   depthHeavyFixture.source,
+  depthHeavyFixture.index,
   depthHeavyFixture.slot,
   dialect,
 );
@@ -70,6 +73,7 @@ if (depthLayout.status !== "ready") {
 }
 const projectedLayout = analyzeSqlCteLayout(
   declarationHeavyFixture.source,
+  declarationHeavyFixture.index,
   declarationHeavyFixture.slot,
   dialect,
 );
@@ -81,6 +85,7 @@ describe("CTE layout", () => {
   bench("ordinary 10 KiB statement", () => {
     analyzeSqlCteLayout(
       ordinaryFixture.source,
+      ordinaryFixture.index,
       ordinaryFixture.slot,
       dialect,
     );
@@ -89,6 +94,7 @@ describe("CTE layout", () => {
   bench("256 declarations", () => {
     analyzeSqlCteLayout(
       declarationHeavyFixture.source,
+      declarationHeavyFixture.index,
       declarationHeavyFixture.slot,
       dialect,
     );
@@ -97,6 +103,7 @@ describe("CTE layout", () => {
   bench("128-depth nested CTE", () => {
     analyzeSqlCteLayout(
       depthHeavyFixture.source,
+      depthHeavyFixture.index,
       depthHeavyFixture.slot,
       dialect,
     );
@@ -105,6 +112,7 @@ describe("CTE layout", () => {
   bench("256 sequential incomplete frames", () => {
     analyzeSqlCteLayout(
       bareFrameHeavyFixture.source,
+      bareFrameHeavyFixture.index,
       bareFrameHeavyFixture.slot,
       dialect,
     );

src/vnext/__tests__/cte-layout.test.ts

@@ -7,6 +7,7 @@ import {
   MAX_CTE_IDENTIFIER_LENGTH,
   MAX_CTE_QUOTED_IDENTIFIER_LENGTH,
   MAX_CTE_STATEMENT_LENGTH,
+  resolveAuthenticatedSqlCteEntrypoints,
   type SqlCteLayout,
   type SqlCteLayoutDialect,
   type SqlCteLayoutIssue,
@@ -55,6 +56,7 @@ function analyze(
   expect(slot?.boundaryQuality).toBe("exact");
   const result = analyzeSqlCteLayout(
     source,
+    index,
     slot as ExactSqlStatementSlot,
     dialect,
   );
@@ -68,15 +70,18 @@ function analyze(
 function analyzeRaw(
   text: string,
   dialect: SqlCteLayoutDialect = postgres,
+  indexDialect: SqlCteLayoutDialect = dialect,
 ): SqlCteLayout {
   const source = createIdentitySqlSource(text);
-  const slot = buildSqlStatementIndex(
+  const index = buildSqlStatementIndex(
     source.analysisText,
-    dialect.lexicalProfile,
-  ).slots[0];
+    indexDialect.lexicalProfile,
+  );
+  const slot = index.slots[0];
   expect(slot?.boundaryQuality).toBe("exact");
   return analyzeSqlCteLayout(
     source,
+    index,
     slot as ExactSqlStatementSlot,
     dialect,
   );
@@ -400,6 +405,7 @@ describe("bounded CTE layout", () => {
     }
     const layout = analyzeSqlCteLayout(
       source,
+      index,
       slot,
       postgres,
     );
@@ -1022,13 +1028,60 @@ describe("bounded CTE layout", () => {
         },
       },
     };
-    expect(analyzeRaw("SELECT 1", getterLexicalProfile)).toEqual({
+    expect(
+      analyzeRaw("SELECT 1", getterLexicalProfile, postgres),
+    ).toEqual({
       reason: "resource-limit",
       status: "unavailable",
     });
     expect(getterInvoked).toBe(false);
   });
 
+  it("never reads a raw dialect after validating its data descriptors", () => {
+    const text = "WITH local AS (SELECT 1) SELECT * FROM local";
+    const source = createIdentitySqlSource(text);
+    const index = buildSqlStatementIndex(
+      source.analysisText,
+      postgres.lexicalProfile,
+    );
+    const slot = index.slots[0];
+    expect(slot?.boundaryQuality).toBe("exact");
+    if (!slot || slot.boundaryQuality !== "exact") {
+      throw new Error("Expected one exact statement slot");
+    }
+
+    let rawRead = false;
+    const descriptorOnlyDialect = new Proxy(postgres, {
+      get(target, property, receiver) {
+        if (property === "lexicalProfile") {
+          rawRead = true;
+          throw new Error("hostile");
+        }
+        return Reflect.get(target, property, receiver);
+      },
+    });
+    const layout = analyzeSqlCteLayout(
+      source,
+      index,
+      slot,
+      descriptorOnlyDialect,
+    );
+
+    expect(layout.status).toBe("ready");
+    if (layout.status === "unavailable") {
+      throw new Error("Expected an authenticated CTE layout");
+    }
+    expect(
+      resolveAuthenticatedSqlCteEntrypoints(
+        layout,
+        source,
+        slot,
+        descriptorOnlyDialect,
+      ),
+    ).toEqual(layout.mainQueryEntrypoints);
+    expect(rawRead).toBe(false);
+  });
+
   it("projects only proven frame phases and validates positions", () => {
     const text =
       "WITH a AS (SELECT 1) SELECT * FROM a";

src/vnext/__tests__/incremental-statement-index.test.ts

@@ -4,6 +4,7 @@ import {
   buildSqlStatementIndex,
   DREMIO_SQL_LEXICAL_PROFILE,
   DUCKDB_SQL_LEXICAL_PROFILE,
+  isExactSqlStatementSlotSnapshotFor,
   MAX_SQL_STATEMENT_SLOTS,
   POSTGRESQL_SQL_LEXICAL_PROFILE,
   type SqlLexicalProfile,
@@ -358,18 +359,93 @@ describe("incremental statement index reuse", () => {
     },
   );
 
+  it("rebuilds when empty change metadata does not match the text", () => {
+    const previousIndex = buildSqlStatementIndex(
+      "SELECT 1",
+      POSTGRESQL_SQL_LEXICAL_PROFILE,
+    );
+    const nextIndex = updateSqlStatementIndex(
+      previousIndex,
+      "SELECT 2",
+      [],
+      POSTGRESQL_SQL_LEXICAL_PROFILE,
+    );
+    expect(nextIndex).not.toBe(previousIndex);
+    expect(nextIndex).toEqual(
+      buildSqlStatementIndex(
+        "SELECT 2",
+        POSTGRESQL_SQL_LEXICAL_PROFILE,
+      ),
+    );
+  });
+
+  it("rebuilds when the lexical profile changes", () => {
+    const text = "SELECT 1";
+    const previousIndex = buildSqlStatementIndex(
+      text,
+      DUCKDB_SQL_LEXICAL_PROFILE,
+    );
+    const nextIndex = updateSqlStatementIndex(
+      previousIndex,
+      text,
+      [],
+      POSTGRESQL_SQL_LEXICAL_PROFILE,
+    );
+    expect(nextIndex).not.toBe(previousIndex);
+    expect(nextIndex).toEqual(
+      buildSqlStatementIndex(
+        text,
+        POSTGRESQL_SQL_LEXICAL_PROFILE,
+      ),
+    );
+  });
+
+  it("rebuilds when one lexical profile object mutates", () => {
+    const profile = { ...POSTGRESQL_SQL_LEXICAL_PROFILE };
+    const text = "SELECT # ;\n SELECT 2";
+    const previousIndex = buildSqlStatementIndex(text, profile);
+    expect(previousIndex.slots).toHaveLength(2);
+    profile.hashLineComments = true;
+    const nextIndex = updateSqlStatementIndex(
+      previousIndex,
+      text,
+      [],
+      profile,
+    );
+    expect(nextIndex).not.toBe(previousIndex);
+    expect(nextIndex).toEqual(buildSqlStatementIndex(text, profile));
+    expect(nextIndex.slots).toHaveLength(1);
+  });
+
   it("reuses unchanged prefix and zero-delta suffix slot identities", () => {
     const profile = DUCKDB_SQL_LEXICAL_PROFILE;
     const text = "SELECT 1; SELECT 2; SELECT 3";
     const change = replaceFirst(text, "2", "9");
-    const { nextIndex, previousIndex } = expectIncrementalMatchesOracle(
-      text,
-      [change],
-      profile,
-    );
-    expect(expectSlot(nextIndex.slots, 0)).toBe(
-      expectSlot(previousIndex.slots, 0),
-    );
+    const { nextIndex, nextText, previousIndex } =
+      expectIncrementalMatchesOracle(
+        text,
+        [change],
+        profile,
+      );
+    const sharedPrefix = expectSlot(nextIndex.slots, 0);
+    expect(sharedPrefix).toBe(expectSlot(previousIndex.slots, 0));
+    expect(sharedPrefix.boundaryQuality).toBe("exact");
+    expect(
+      isExactSqlStatementSlotSnapshotFor(
+        previousIndex,
+        sharedPrefix,
+        text,
+        profile,
+      ),
+    ).toBe(true);
+    expect(
+      isExactSqlStatementSlotSnapshotFor(
+        nextIndex,
+        sharedPrefix,
+        nextText,
+        profile,
+      ),
+    ).toBe(true);
     expect(expectSlot(nextIndex.slots, 1)).not.toBe(
       expectSlot(previousIndex.slots, 1),
     );