feat(vnext): validate relation catalog data

Author: Light2Dark

docs/adr/0005-parser-independent-relation-completion.md

@@ -347,7 +347,13 @@ search request contains only copied, recursively frozen plain data:
 The `AbortSignal` is passed separately. Providers never receive a session,
 revision, `EditorState`, DOM object, credential object, or arbitrary live host
 context. Connection identity belongs in the catalog scope; live resources stay
-inside the provider closure.
+inside the provider closure. The provider object is caller-owned. The service
+captures its own-data callbacks once and owns searches, subscriptions, and
+their cleanup, but does not imply provider-wide disposal. Provider callbacks
+are closure functions with a declared `this: void` contract and are invoked
+with `this === undefined`; mutable receiver state is not part of the provider
+API. Unrelated own configuration fields are ignored, while inherited callbacks
+and accessors are rejected.
 
 A returned relation contains:
 
@@ -363,7 +369,10 @@ A returned relation contains:
 
 The initial closed component roles are `catalog`, `schema`, `project`,
 `dataset`, and `relation`. Each dialect accepts only its documented role
-sequences, and the final component is always `relation`.
+sequences, and the final component is always `relation`. The provider's
+`quoted` bit is positive catalog evidence that the canonical decoded spelling
+is quoted; it is never insertion SQL. The dialect still owns delimiters,
+escaping, reserved-word quoting, and rendering.
 
 The service never invents an unqualified candidate from an absolute path. The
 provider proves matching and addressability through `matchQuality` and
@@ -397,7 +406,25 @@ as an unknown-object diagnostic.
 Provider responses are decoded from `unknown` using bounded own enumerable
 data properties and copied into fresh frozen current-realm objects. Accessors,
 throwing proxies, unexpected keys, oversized strings or arrays, and malformed
-closed values fail without exposing raw errors.
+closed values fail without exposing raw errors. One malformed relation rejects
+the whole page: silently retaining other entities while preserving the
+provider's coverage claim would create false authority. Entity IDs must be
+unique within a page. Their stability across searches and epochs is a provider
+promise used for provenance, not authority the service can infer by generic
+deduplication. Shared input identities are allowed, but each occurrence is
+decoded, budgeted, and copied independently, so aliasing cannot retain mutable
+provider state or bypass aggregate limits.
+
+Both the complete canonical role sequence and the suffix selected by
+`completionPathStart` must be legal for the registered dialect. The boundary
+pre-renders the selected suffix from fresh decoded data so later scheduling and
+composition never touch raw provider objects or provider-supplied SQL.
+
+The initial vertical slice validates one bounded page and reports paginated
+coverage as incomplete; it does not automatically follow or merge continuation
+tokens. Continuation tokens are opaque provider state. They are bounded,
+retained only with their page/work state, and never included in completion
+items, logs, errors, or revision events.
 
 The core also provides a provisional `createInMemoryRelationCatalog` helper for
 bounded readonly relation data. It indexes stable IDs, kinds, role-bearing
@@ -439,8 +466,20 @@ A search that discovers a higher epoch supersedes itself instead of publishing
 against its older captured revision. Pages and cache entries from different
 epochs are never merged.
 
+Any successfully decoded response status (`ready`, `loading`, or `failed`) may
+establish the first epoch baseline. Equal-epoch responses for different exact
+searches remain usable; only the epoch observation is a duplicate. A terminal
+`loading` response requires a higher epoch before that exact search can become
+ready. Providers without subscriptions may return `loading`, but they cannot
+cause automatic refresh; a later explicit request must observe the higher
+epoch. Failed responses are attempt evidence: `next-request` may recover at
+the same epoch, while `after-invalidation` and `never` create bounded retry
+gates rather than cached search results.
+
 The service reference-counts one provider subscription per provider
 configuration and scope, then fans invalidation out to subscribed sessions.
+The installed callback closure supplies authenticated provider and scope
+identity; the untrusted invalidation payload therefore contains only an epoch.
 Subscription membership is installed atomically before a provider can call
 back synchronously. A newly joining session captures an already-observed epoch
 without a synthetic revision bump. Disposal removes membership before
@@ -639,7 +678,7 @@ The initial checked limits are:
 | Plain-text detail | 1,024 UTF-16 units |
 | Continuation token | 2,048 UTF-16 units |
 | Decoded response aggregate | 65,536 UTF-16 units |
-| Decoded response own keys | 1,024 |
+| Decoded response own keys | 16,384 aggregate traversal occurrences |
 | Decoded response nesting depth | 8 |
 | Service catalog cache | 256 entries and 2 MiB estimated retained bytes |
 | Service-wide active catalog searches | 8 |

package.json

@@ -26,6 +26,7 @@
     "test:browser": "vitest run --config vitest.browser.config.ts",
     "test:worker-placement": "node ./scripts/worker-placement.mjs",
     "test:integrity": "node ./scripts/check-test-integrity.mjs",
+    "bench:catalog-boundary": "vitest bench --run src/vnext/__tests__/relation-catalog-boundary.bench.ts",
     "bench:local-relation-site": "vitest bench --run src/vnext/__tests__/local-relation-site.bench.ts",
     "bench:parser-adapter": "vitest bench --run src/vnext/__tests__/node-sql-parser-adapter.bench.ts",
     "bench:query-site": "vitest bench --run src/vnext/__tests__/query-site.bench.ts",

src/vnext/__tests__/relation-catalog-boundary.bench.ts

@@ -0,0 +1,153 @@
+import { bench, describe } from "vitest";
+import { decodeSqlCatalogSearchResponse } from "../relation-catalog-boundary.js";
+import {
+  DREMIO_SQL_RELATION_DIALECT,
+  POSTGRESQL_SQL_RELATION_DIALECT,
+} from "../relation-dialect.js";
+
+function relation(index: number) {
+  return {
+    canonicalPath: [
+      { quoted: false, role: "schema", value: "public" },
+      {
+        quoted: false,
+        role: "relation",
+        value: `relation_${index}`,
+      },
+    ],
+    completionPathStart: 0,
+    detail: `Relation ${index}`,
+    entityId: `relation-${index}`,
+    matchQuality: "exact",
+    relationKind: "table",
+  };
+}
+
+function response(relations: readonly unknown[]) {
+  return {
+    coverage: { kind: "complete" },
+    epoch: { generation: 1, token: "snapshot-1" },
+    relations,
+    status: "ready",
+  };
+}
+
+const emptyResponse = response([]);
+const oneResponse = response([relation(0)]);
+const hundredResponse = response(
+  Array.from({ length: 100 }, (_, index) => relation(index)),
+);
+const malformedLastResponse = response([
+  ...Array.from({ length: 99 }, (_, index) => relation(index)),
+  { ...relation(99), completionPathStart: 9 },
+]);
+const oversizedTextResponse = response([
+  { ...relation(0), detail: "x".repeat(1_000_000) },
+]);
+const maximumDepthResponse = response(
+  Array.from({ length: 7 }, (_, relationIndex) => ({
+    ...relation(relationIndex),
+    canonicalPath: Array.from({ length: 32 }, (_, pathIndex) => ({
+      quoted: true,
+      role: pathIndex === 31 ? "relation" : "schema",
+      value: String(relationIndex).repeat(256),
+    })),
+    detail: "d".repeat(1_024),
+    entityId: `maximum-depth-${relationIndex}`,
+  })),
+);
+const wideInvalidResponse = Object.fromEntries(
+  Array.from({ length: 10_000 }, (_, index) => [
+    `unexpected-${index}`,
+    index,
+  ]),
+);
+
+for (const [candidate, expectedStatus] of [
+  [emptyResponse, "accepted"],
+  [oneResponse, "accepted"],
+  [hundredResponse, "accepted"],
+  [malformedLastResponse, "malformed"],
+  [oversizedTextResponse, "malformed"],
+] as const) {
+  const result = decodeSqlCatalogSearchResponse(
+    candidate,
+    100,
+    POSTGRESQL_SQL_RELATION_DIALECT,
+  );
+  if (result.status !== expectedStatus) {
+    throw new Error("Catalog boundary benchmark preflight failed");
+  }
+}
+if (
+  decodeSqlCatalogSearchResponse(
+    maximumDepthResponse,
+    100,
+    DREMIO_SQL_RELATION_DIALECT,
+  ).status !== "accepted" ||
+  decodeSqlCatalogSearchResponse(
+    wideInvalidResponse,
+    100,
+    POSTGRESQL_SQL_RELATION_DIALECT,
+  ).status !== "malformed"
+) {
+  throw new Error("Catalog boundary benchmark preflight failed");
+}
+
+describe("relation catalog boundary", () => {
+  bench("decode empty ready response", () => {
+    decodeSqlCatalogSearchResponse(
+      emptyResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("decode one relation", () => {
+    decodeSqlCatalogSearchResponse(
+      oneResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("decode 100 relations", () => {
+    decodeSqlCatalogSearchResponse(
+      hundredResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("reject malformed final relation", () => {
+    decodeSqlCatalogSearchResponse(
+      malformedLastResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("reject a million-unit detail before scanning it", () => {
+    decodeSqlCatalogSearchResponse(
+      oversizedTextResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("decode near-limit maximum-depth relations", () => {
+    decodeSqlCatalogSearchResponse(
+      maximumDepthResponse,
+      100,
+      DREMIO_SQL_RELATION_DIALECT,
+    );
+  });
+
+  bench("reject a wide unexpected record", () => {
+    decodeSqlCatalogSearchResponse(
+      wideInvalidResponse,
+      100,
+      POSTGRESQL_SQL_RELATION_DIALECT,
+    );
+  });
+});