feat(vnext): complete SQL editor overhaul (#202)

## Summary

Completes the final batched vNext SQL language-service overhaul
milestone.

- adds authenticated parser-neutral query binding models and bounded
  `node-sql-parser` normalization in the isolated worker protocol
- adds public batched column and namespace catalog providers with stable
provenance, epochs, bounded caches, cancellation,
partial/loading/failure
  states, and hostile-response validation
- integrates relation, namespace, and physical-column completion into
document
  sessions and the CodeMirror adapter
- adds exact statement gutter behavior and insertion-point completion
gating
  for embedded marimo expressions while preserving external sources
- adds correlated outer-scope column completion with conservative
derived-table
  isolation
- expands the marimo compile-time migration fixture and
architecture/authority
  documentation

## Review hardening

All 27 Cubic comments were reproduced and addressed. Two final
adversarial
reviews found and closed seven additional issues:

- revoked proxies, decoder exceptions, exact response variants,
structural
  identity comparisons, and blank namespace path components
- generation-linearized cancellation and disposal in both catalog
coordinators,
  including reentrant abort listeners
- source-authenticated identifiers, aliases, CTE scope, derived ranges,
compound
  blocks, and `ON`/`USING` visibility
- exact statement bounds, one end-to-end response budget, the
64-relation
deterministic partial cap, incomplete aliases, and line-comment
boundaries
- stale completion-gate callbacks, external-source preservation, and
  failure-safe browser cleanup

CTE and derived-table output-column inference remains an explicit
next-major
cutover gap; vNext does not incorrectly route those logical relations to
the
physical catalog provider.

## Correctness and performance

- original-document UTF-16 ranges and exact replacement edits
- explicit partial/unsupported results instead of guessed semantics
- revision, cancellation, owner disposal, scope, dialect, and epoch
checks
- no raw parser AST crosses or persists beyond the adapter/worker
boundary
- complete-only bounded catalog caches
- deterministic ordering and deduplication
- maximum-shape query-model validation reduced from about 888 ms to
about
  6.4 ms
- framework-independent packed core: 187,989 raw bytes and 50,590 gzip
bytes,
  within the unchanged 184 KiB raw and 50 KiB gzip limits

## Verification

- 2,286 unit tests passed; 1 intentional expected failure
- repository coverage:
  - 96.58% statements
  - 96.71% lines
  - 98.01% functions
  - 94.43% branches
- changed vNext coverage:
  - 97.79% statements
  - 98.08% lines
  - 99.46% functions
  - 96.62% branches
- every changed runtime file passes the 95% per-file
  statements/branches/functions/lines gate
- strict TypeScript configurations, oxlint, and test-integrity checks
pass
- 14 browser tests pass in Chromium
- build, demo build, packed-consumer smoke, parser-isolation/CSP checks,
and
  isolated-worker placement audit pass

Advances the next-major implementation milestone tracked in #169.

Author: Light2Dark

docs/adr/0003-node-sql-parser-adapter.md

@@ -145,11 +145,11 @@ A direct object or one-element array is accepted. Zero or multiple roots are
 `failed/malformed-output`.
 
 The public normalized artifact exposes only its closed statement kind and full
-statement-relative range. The backend AST is retained in a module-private
-`WeakMap` keyed by the authenticated artifact. It is neither enumerable nor
-returned through the syntax contract. Future relation extraction must decode
-and validate backend nodes inside the adapter boundary rather than exposing
-the raw AST to core features.
+statement-relative range. The adapter consumes the backend AST immediately
+through the bounded query-binding decoder and retains only an immutable,
+backend-neutral binding model in a module-private `WeakMap` keyed by the
+authenticated artifact. The raw AST is not retained, enumerable, or returned
+through the syntax contract.
 
 ### Cancellation and execution placement
 
@@ -201,7 +201,7 @@ This decision does not define:
 - Stable parser or provider APIs
 - Session cache keys or eviction
 - Document-level syntax diagnostics
-- Semantic relation, scope, column, or type models
+- Public semantic relation, scope, column, or type models
 - A worker protocol
 - Native DuckDB or remote validation providers
 - Dremio parsing

docs/adr/0004-isolated-parser-execution.md

@@ -141,9 +141,10 @@ The initial request contains only:
 DuckDB uses the PostgreSQL grammar. Target-dialect policy stays in the main
 realm.
 
-The initial response contains only one closed outcome:
+Protocol v2 contains only one closed outcome:
 
-- Parsed normalized statement kind
+- Parsed normalized statement kind and a bounded query-binding DTO, or `null`
+  when the statement has no supported query-binding evidence
 - Syntax rejection
 - Bounded unsupported reason
 - Bounded failure code; retryability is derived from that code
@@ -183,16 +184,17 @@ settles the active operation exactly once without exposing raw event data.
 Raw backend ASTs will not cross the worker boundary and the first protocol will
 not introduce remote AST handles or worker-local AST leases.
 
-The first relation-completion slice is parser-independent under
+The first relation-completion slice remains parser-independent under
 [ADR 0005](0005-parser-independent-relation-completion.md). Protocol v1 remains
-syntax-only. Incomplete relation completion must not wait for worker startup,
+historical and protocol v2 adds the internal query-binding DTO. Incomplete
+relation completion must not wait for worker startup,
 parser acceptance, timeout, or recovery.
 
-A future scope-dependent semantic slice will parse once and run adapter-owned
-semantic decoders in the worker realm. It will move the closed protocol
-atomically to v2 and return a bounded scoped IR, with normalized syntax and
-semantic availability represented independently. It will not return flat
-relation names, raw ASTs, source text, or absolute document ranges.
+The scope decoder runs once in the worker realm immediately after parsing.
+It returns query blocks, relation/alias bindings, persistent scopes, typed
+visibility regions, independent coverage, and closed issues. It does not
+return flat relation names, raw ASTs, source text, or absolute document ranges.
+The host revalidates the complete DTO before accepting the response.
 
 Worker-local AST caching is deferred until profiling demonstrates that
 reparsing is material enough to justify leases, byte accounting, generation

docs/adr/0006-query-binding-model.md

@@ -0,0 +1,103 @@
+# ADR 0006: Internal query binding model
+
+## Status
+
+Accepted for the vNext implementation. The model remains package-internal until a
+column-completion vertical slice has validated it across the supported dialect
+corpora.
+
+## Context
+
+Column completion needs more than a parser's flat table list. It must distinguish
+query blocks, relation declarations, aliases, correlation, and clause-specific
+visibility. The parser adapter also has strict trust boundaries: raw parser ASTs
+are private, parser results can be partial, and cached analysis must belong to
+the exact statement and parser authority that produced it.
+
+The model must remain cheap enough for interactive use and must not make the
+existing parser-independent relation completion path wait for parsing.
+
+## Decision
+
+Add an internal, backend-neutral query binding model with:
+
+- statement-relative UTF-16 ranges;
+- query blocks and relation bindings identified by model-local array indexes;
+- a persistent scope chain in which each node adds at most one binding;
+- ordered, globally non-overlapping visibility regions that point to a scope;
+- independent query-block, relation-binding, and visibility coverage;
+- closed issue and unknown-source reason sets;
+- private authentication and exact statement/parser-authority ownership.
+
+Globally non-overlapping regions deliberately exclude nested query text from
+their parent region. A decoder splits a parent region around nested queries.
+This permits binary-search lookup without duplicating the visible binding set.
+Persistent scopes then reconstruct bindings in declaration order using linear
+space. In particular:
+
+- a `SELECT` list can point to the completed `FROM` scope even though it appears
+  earlier in the source;
+- a join condition points to the scope containing prior relations and its
+  current right-hand relation;
+- a correlated child block can enter through a parent scope;
+- an ordinary derived table enters through a scope that excludes same-level
+  siblings; a future proven `LATERAL` decoder may opt into that scope.
+
+Aliases are the visible qualifier when present and hide a named relation's base
+name. Identifier equality is supplied by the dialect runtime; the model does
+not lowercase or otherwise reinterpret identifiers.
+
+Construction accepts untrusted plain data and validates it before issuing an
+immutable authenticated model. It rejects accessors, sparse or oversized
+arrays, malformed UTF-16 identifiers, invalid or unrelated ranges, forward
+parent edges, repeated bindings in a scope chain, overlapping regions, and
+unknown bindings paired with complete relation coverage.
+
+Resolution returns:
+
+- a visible binding list with complete or partial coverage;
+- one resolved binding, known ambiguity, or authoritative no-match;
+- unavailable instead of no-match when coverage is partial.
+
+## Ownership and invalidation
+
+The future parser service owns bounded parse/model caching and in-flight
+deduplication. A document session owns the mapping from its current statement
+slots to immutable models. The cache key includes exact statement text,
+dialect, parser authority/conformance, and implementation version.
+Statement-relative coordinates allow an unchanged statement to be reused after
+an edit shifts its absolute document position. Source, dialect, or parser
+authority changes invalidate the model; catalog epoch and completion context
+changes do not.
+
+The interactive statement limit is initially 16 KiB. The model also caps query
+blocks at 256, relation bindings at 1,024, scopes and regions at 2,048 each,
+nesting by the block limit, issues at 256, path depth at 8, and identifier
+components at 256 UTF-16 code units.
+
+## Degradation
+
+A decoder may claim complete evidence only for directly conforming parser output
+and constructs it understands. Compatibility parses can supply useful partial
+evidence but cannot prove absence. Unknown AST subtrees become an unknown source
+or a closed issue and downgrade the relevant coverage; they are never silently
+dropped under complete coverage.
+
+Invalid, failed, opaque, incomplete, or resource-limited analysis becomes
+partial or unavailable. Parser-independent relation completion remains
+available. Dremio remains partial or unavailable until it has an owned semantic
+corpus.
+
+## Consequences
+
+The model can support lazy batched column loading without exposing a parser AST
+or CodeMirror's eager `SQLNamespace`. Its indexes are not stable identities
+across revisions. Projection inference, output columns, star expansion, types,
+expression binding, semantic diagnostics, navigation, rename, formatting, and
+vendor constructs such as `PIVOT`, `UNNEST`, table functions, and unproven
+`LATERAL` semantics are explicitly outside this slice.
+
+The node-sql-parser integration now supplies a strict AST-to-plain-data decoder,
+normalizes inside the worker, and checks direct-versus-worker parity. Public
+semantic APIs remain deferred until a column-completion vertical slice proves
+the model against consumer and dialect corpora.

docs/vnext/capability-charter.md

@@ -239,7 +239,8 @@ count and estimated retained bytes.
 
 Provisional compressed bundle budgets:
 
-- Framework-independent core with relation completion: 48 KiB
+- Framework-independent core with relation, namespace, and column completion:
+  50 KiB
 - Core plus CodeMirror adapter, excluding parser and dialect data: 75 KiB
 - Each ordinary dialect module: 25 KiB
 - Optional `node-sql-parser` chunk: no regression from its recorded baseline

docs/vnext/codemirror-adapter.md

@@ -70,6 +70,51 @@ const support = sqlEditor({
 });
 ```
 
+## Statement boundaries and gutter
+
+The support object exposes synchronous structural queries without exposing the
+adapter-owned session:
+
+```ts
+const current = support.statementBoundaryAt(view, {
+  affinity: "left",
+  position: view.state.selection.main.head,
+});
+const visible = support.statementBoundariesIntersecting(view, {
+  from: view.viewport.from,
+  to: view.viewport.to,
+});
+```
+
+Both methods return `null` for foreign, destroyed, or unsynchronized views.
+Catalog-backed hosts can invalidate the adapter-owned session without exposing
+it:
+
+```ts
+const revision = support.invalidateCatalog(view);
+```
+
+This returns `null` for foreign or destroyed views. A live view advances its
+revision, cancels stale catalog work, and forces relation, column, and namespace
+providers to be consulted again.
+
+An opt-in structural gutter marks only lines intersecting scanner-owned SQL
+`code` spans. It never parses or copies statement text:
+
+```ts
+const support = sqlEditor({
+  initialContext,
+  service,
+  statementGutter: {
+    hideWhenNotFocused: true,
+    showInactive: true,
+  },
+});
+```
+
+The gutter uses `--cm-sql-statement-color` and
+`--cm-sql-statement-inactive-opacity` CSS variables. It is disabled by default.
+
 ## Atomic inputs
 
 Context and document changes can share one CodeMirror transaction:

docs/vnext/column-catalog-authority.md

@@ -0,0 +1,65 @@
+# vNext Column Catalog Authority
+
+Status: internal vertical-slice contract
+
+Column discovery is lazy, provider-owned, and batched. A completion request
+sends at most 64 unresolved relation references in one provider invocation.
+The deterministic first batch is useful but explicitly partial when more
+visible relations match; completion reports `query-binding-partial` and never
+claims the omitted relations were searched. Each reference carries a
+caller-local `requestKey`, a decoded identifier path, and no unauthenticated
+entity identity. The provider resolves paths against the supplied catalog
+scope, search paths, and dialect.
+
+The provider returns stable relation and column entity IDs. Every accepted
+column contains:
+
+- a canonical `SqlIdentifierComponent`;
+- bounded provider-rendered `insertText`;
+- a stable column entity ID and ordinal; and
+- immutable provenance containing provider, scope, epoch, relation, and column
+  identities.
+
+Responses describe each requested relation independently as:
+
+- `ready` with complete or partial column coverage;
+- `loading`; or
+- `failed` with a normalized code and retry policy.
+
+Missing, extra, conflicting, oversized, accessor-backed, or malformed data is
+rejected at the provider boundary. Relations and columns have deterministic
+code-unit order. Duplicate request keys and conflicting stable IDs fail closed;
+identical duplicate columns are deduplicated.
+
+## Epoch and cache behavior
+
+Cold requests use `expectedEpoch: null`. A response always supplies an epoch.
+An owner remembers that observation, and later null-epoch requests reuse only
+cache entries from the observed epoch. Explicit expected epochs never reuse
+entries from another epoch. The cache is LRU-bounded by relation entries.
+Only complete ready results are cached. Partial, loading, and failed results
+remain visible to the caller but are eligible for another provider request.
+
+Relation-catalog subscription events invalidate the session's relation, column,
+and namespace observations together. Hosts without a subscribed relation
+provider call `session.invalidateCatalog()` when schema authority changes. The
+method advances the session revision, cancels retained completion work, rebuilds
+all catalog owners, and emits one `catalog` change event.
+
+## Lifecycle
+
+One owner represents one document/session authority for a scope and dialect.
+Starting a newer request supersedes and aborts that owner's prior request.
+Explicit cancellation settles as cancelled. Owner or coordinator disposal
+aborts outstanding work and settles it as unavailable/disposed. Provider
+rejections, throws, and malformed responses are contained; late settlements
+cannot publish or populate the cache.
+
+The coordinator batches a request once, never once per relation. Cache hits and
+misses are composed deterministically while only misses are sent to the
+provider.
+
+Provider-declared loading receives at most one automatic retry for the same
+document, context, and completion position. A persistent loading response is
+then returned without a refresh token; hosts resume it through catalog
+invalidation instead of an unbounded polling loop.