feat(vnext): add source coordinate primitives (#173)

## Summary

- publish the minimal `SqlTextRange` UTF-16 contract and make
`SqlTextChange` extend it
- add internal immutable identity/masked source snapshots with explicit
`originalText` and `analysisText`
- validate and freeze bounded embedded regions without invoking
accessors or Proxy `get` traps
- preserve length and every CR/LF offset while masking every other
UTF-16 code unit
- store source snapshots in document sessions, reusing them only for
context-only updates
- document the intentional boundary: no public transformer or source-map
SPI yet
- request Copilot once per PR, without later re-requests

## Invariants and risk

- ranges are safe-integer, in-bounds, half-open UTF-16 offsets
- embedded regions are non-empty, ordered, non-overlapping, and capped
at 10,000
- source text is capped at 16 Mi UTF-16 code units
- masking uses bounded 64 Ki-code-unit chunks and cannot allocate per
newline
- arbitrary values thrown by hostile Proxy traps are normalized without
inspecting them
- document updates create a new source snapshot; context-only updates
retain the exact source identity

## Evidence

- `pnpm run typecheck`
- `pnpm exec oxlint`
- `pnpm run test:coverage` (713 passed, 1 governed expected failure)
- changed runtime coverage: 98.78% statements, 97.75% branches, 100%
functions, 98.76% lines
- deterministic randomized UTF-16 masking/range round trips
- full 16,777,216-code-unit CRLF mask and 10,000-region smoke cases
- full-limit newline mask succeeds under `node --max-old-space-size=128`
- `pnpm run test:browser`
- `pnpm run test:package`
- `pnpm run test:integrity`
- `pnpm run demo`

## Adversarial review

Two independent exact-head reviewers challenged correctness and
performance/API design. The first loop found a newline-density memory
amplification and hostile thrown-Proxy escape. Both were redesigned and
regression-tested. Both reviewers approve exact SHA
`1993a6cdca85ec9d3b8a1ddaab5dda0188b85fab` with no blocker, high, or
medium findings.


<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Add internal source coordinate primitives and length-preserving masking
to support safe UTF-16 ranges and embedded-region analysis. Export
`SqlTextRange` and update sessions to use immutable source snapshots,
reusing them for context-only updates.

- New Features
  - Export `SqlTextRange`; make `SqlTextChange` extend it.
- Add immutable source snapshots with `originalText` and `analysisText`,
plus length-preserving masking of embedded regions (CR/LF offsets
preserved). No transformer or source-map SPI yet.
- Validate and freeze inputs with strict caps (16 Mi code units, 10k
regions) and accessor-safe normalization.
- Store source snapshots in document sessions and reuse them on
context-only updates.

<sup>Written for commit 1993a6c.
Summary will update on new commits.</sup>

<a
href="https://cubic.dev/pr/marimo-team/codemirror-sql/pull/173?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>

<!-- End of auto-generated description by cubic. -->

Author: Light2Dark

docs/adr/0001-language-service-and-session.md

@@ -389,6 +389,12 @@ adjacent SQL tokens. More complex transformers use a versioned map whose
 segments are ordered, non-overlapping, and validated for original/generated
 coverage.
 
+The initial internal source primitive implements identity snapshots and this
+length-preserving masking only. It does not expose a transformer or source-map
+SPI. Sessions create a new source snapshot for document updates and reuse the
+existing snapshot for context-only updates. Non-identity generated/reordered
+source remains deferred until a concrete consumer validates the segment model.
+
 Edits crossing an ambiguous or unmapped boundary are rejected. Mapping failure
 is unavailable analysis, not invalid SQL.
 

docs/vnext/session-primitives.md

@@ -8,6 +8,9 @@ updates, opaque revisions, dialect registration, and lifecycle management. It
 does not yet provide parsing, completion, diagnostics, hover, or navigation.
 Those methods will be added only with working vertical slices.
 
+See [source coordinates](./source-coordinates.md) for the shared UTF-16 range
+contract and the internal immutable source-snapshot model.
+
 ## Example
 
 ```ts

docs/vnext/source-coordinates.md

@@ -0,0 +1,46 @@
+# vNext Source Coordinates
+
+Status: experimental core primitive
+
+`SqlTextRange` is the public coordinate primitive:
+
+```ts
+interface SqlTextRange {
+  readonly from: number;
+  readonly to: number;
+}
+```
+
+Ranges are half-open UTF-16 offsets. Both ends are safe integers and satisfy
+`0 <= from <= to <= document.length`. Empty ranges, including the range at EOF,
+are valid. `SqlTextChange` extends this shape with `insert` and continues to use
+pre-update document coordinates.
+
+The service internally owns an immutable source snapshot with separately named
+`originalText` and `analysisText`. A document update creates a new identity
+snapshot; a context-only update reuses the same source object. This separation
+allows later analysis transforms without making providers depend on editor
+state or ambiguous coordinate spaces.
+
+## Length-preserving masking
+
+The first internal transform masks explicit embedded regions:
+
+- Regions are non-empty, ordered, non-overlapping, and in bounds.
+- Region and range inputs are copied into fresh frozen values.
+- Each non-CR/LF UTF-16 code unit becomes one space.
+- Every CR and LF code unit stays at its exact offset.
+- Astral characters therefore become two spaces, while a lone surrogate becomes
+  one space.
+- `analysisText.length` always equals `originalText.length`, so range mapping is
+  identity-preserving while still returning a fresh validated range.
+
+At most 10,000 regions and 16 Mi UTF-16 code units are accepted. Masking uses
+bounded 64 Ki-code-unit chunks, including for newline-dense input, rather than
+a per-code-unit or per-newline array.
+
+The source snapshot, embedded-region model, masking factory, and mapping
+functions remain internal. This slice intentionally does not publish a source
+transformer or source-map SPI. Generated or reordered source requires a
+versioned segment-map design and evidence from real consumers before becoming
+public.

implementation.md

@@ -590,8 +590,8 @@ disposition. Validate them through a protected reusable workflow from the base
 branch so a PR cannot approve itself. Maintainers approve classification
 overrides and rejected blocking findings.
 
-Request GitHub Copilot review on every medium or large PR after the head commit
-is ready for review:
+Request GitHub Copilot review exactly once on every medium or large PR, after
+the PR has a coherent head that is ready for review:
 
 ```bash
 gh pr edit <number> --add-reviewer @copilot
@@ -600,8 +600,9 @@ gh pr edit <number> --add-reviewer @copilot
 Copilot is an additional advisory review, not one of the two independent
 commit-bound adversarial attestations and not a substitute for human approval.
 Resolve or explicitly disposition its actionable comments in the finding
-ledger. Re-request review after a material rewrite when the prior comments no
-longer cover the current head.
+ledger. Do not re-request Copilot after later pushes or material rewrites; the
+commit-bound adversarial reviewers and required CI provide exact-head
+revalidation.
 
 ## CI architecture
 
@@ -814,7 +815,7 @@ Before semantic implementation:
 12. Add the invariant registry and test-suite integrity checks.
 13. Add PR risk template and review-packet generation.
 14. Add protected, commit-bound two-reviewer checks.
-15. Automate GitHub Copilot review requests for medium and large PRs.
+15. Automate one GitHub Copilot review request per medium or large PR.
 16. Add verified-artifact provenance to release CI.
 17. Add nightly mutation, fuzz, cross-browser, leak, and performance workflows.
 18. Add expiring scheduled-health and deduplicated failure issues.

scripts/package-smoke.mjs

@@ -210,6 +210,7 @@ import {
   createSqlLanguageService,
   defineSqlDialect,
   type SqlDocumentContext,
+  type SqlTextRange,
 } from "@marimo-team/codemirror-sql/vnext";
 import commonKeywords from "@marimo-team/codemirror-sql/data/common-keywords.json" with { type: "json" };
 import duckdbKeywords from "@marimo-team/codemirror-sql/data/duckdb-keywords.json" with { type: "json" };
@@ -230,6 +231,7 @@ const session = service.openDocument({
   context: { dialect: "duckdb", engine: "local" },
   text: "SELECT 1",
 });
+const range: SqlTextRange = { from: 0, to: 6 };
 session.update({
   kind: "document",
   baseRevision: session.revision,
@@ -238,6 +240,7 @@ session.update({
 
 void extensions;
 void parser;
+void range;
 void session;
 void BigQueryDialect;
 void DremioDialect;

src/vnext/__tests__/session.test.ts

@@ -187,6 +187,28 @@ describe("document revisions", () => {
     });
     expect(second).not.toBe(first);
   });
+
+  it("reuses source snapshots only for context-only updates", () => {
+    const { session } = openSession("SELECT 1");
+    const initialSource = session.snapshotForTesting.source;
+    expect(Object.isFrozen(initialSource)).toBe(true);
+    expect(initialSource.analysisText).toBe(initialSource.originalText);
+
+    session.update({
+      kind: "context",
+      baseRevision: session.revision,
+      context: { dialect: "duckdb", engine: "remote" },
+    });
+    expect(session.snapshotForTesting.source).toBe(initialSource);
+
+    session.update({
+      kind: "document",
+      baseRevision: session.revision,
+      document: { kind: "changes", changes: [] },
+    });
+    expect(session.snapshotForTesting.source).not.toBe(initialSource);
+    expect(session.snapshotForTesting.source.originalText).toBe("SELECT 1");
+  });
 });
 
 describe("document changes", () => {
@@ -204,7 +226,7 @@ describe("document changes", () => {
       },
     });
 
-    expect(session.snapshotForTesting.text).toBe(
+    expect(session.snapshotForTesting.source.originalText).toBe(
       "SELECT customers.id FROM customers",
     );
   });
@@ -219,7 +241,7 @@ describe("document changes", () => {
         changes: [{ from: 1, insert: "X", to: 3 }],
       },
     });
-    expect(session.snapshotForTesting.text).toBe("AXB");
+    expect(session.snapshotForTesting.source.originalText).toBe("AXB");
   });
 
   it("keeps same-position insertions ordered", () => {
@@ -235,7 +257,7 @@ describe("document changes", () => {
         ],
       },
     });
-    expect(session.snapshotForTesting.text).toBe("A12B");
+    expect(session.snapshotForTesting.source.originalText).toBe("A12B");
   });
 
   it("supports adjacent replacements and document boundaries", () => {
@@ -253,7 +275,7 @@ describe("document changes", () => {
         ],
       },
     });
-    expect(session.snapshotForTesting.text).toBe("XYZ!");
+    expect(session.snapshotForTesting.source.originalText).toBe("XYZ!");
   });
 
   it("deletes the entire document", () => {
@@ -263,7 +285,7 @@ describe("document changes", () => {
       baseRevision: session.revision,
       document: { kind: "changes", changes: [{ from: 0, insert: "", to: 8 }] },
     });
-    expect(session.snapshotForTesting.text).toBe("");
+    expect(session.snapshotForTesting.source.originalText).toBe("");
   });
 
   it("allows edits at every UTF-16 boundary", () => {
@@ -280,7 +302,9 @@ describe("document changes", () => {
         ],
       },
     });
-    expect(session.snapshotForTesting.text).toBe("A\uD83DXe\u0301\nZ!\uD800");
+    expect(session.snapshotForTesting.source.originalText).toBe(
+      "A\uD83DXe\u0301\nZ!\uD800",
+    );
   });
 
   it.each([
@@ -305,7 +329,7 @@ describe("document changes", () => {
     });
 
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
   });
 
   it("rejects unordered and overlapping changes atomically", () => {
@@ -325,7 +349,7 @@ describe("document changes", () => {
       });
     });
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("ABCDE");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABCDE");
   });
 
   it("rejects non-string inserts from JavaScript callers", () => {
@@ -356,7 +380,7 @@ describe("document changes", () => {
       session.update({ kind: "document", baseRevision: revision, document } as never);
     });
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
   });
 
   it("rejects an empty JavaScript update", () => {
@@ -423,7 +447,7 @@ describe("document changes", () => {
       baseRevision: session.revision,
       document: { kind: "replace", text: "recovered" },
     });
-    expect(session.snapshotForTesting.text).toBe("recovered");
+    expect(session.snapshotForTesting.source.originalText).toBe("recovered");
   });
 
   it.each([null, "invalid", 42])(
@@ -454,7 +478,7 @@ describe("document changes", () => {
       session.update(accessor as never);
     });
     expect(invoked).toBe(false);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
   });
 
   it("rejects undefined and accessor optional update fields", () => {
@@ -469,7 +493,7 @@ describe("document changes", () => {
       } as never);
     });
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
 
     let invoked = false;
     const update = {
@@ -486,7 +510,7 @@ describe("document changes", () => {
     });
     expect(invoked).toBe(false);
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
   });
 
   it("rejects accessor document fields without invoking them", () => {
@@ -571,7 +595,7 @@ describe("document changes", () => {
 
     expect(nestedError?.code).toBe("reentrant-update");
     expect(session.isCurrent(revision)).toBe(true);
-    expect(session.snapshotForTesting.text).toBe("outer");
+    expect(session.snapshotForTesting.source.originalText).toBe("outer");
   });
 
   it("cannot commit an update after reentrant disposal", () => {
@@ -600,7 +624,7 @@ describe("document changes", () => {
       });
     });
     expect(session.isCurrent(revision)).toBe(false);
-    expect(session.snapshotForTesting.text).toBe("ABC");
+    expect(session.snapshotForTesting.source.originalText).toBe("ABC");
   });
 
   it("bounds fragmented and oversized document updates", () => {
@@ -631,7 +655,7 @@ describe("document changes", () => {
         },
       });
     });
-    expect(session.snapshotForTesting.text).toBe("");
+    expect(session.snapshotForTesting.source.originalText).toBe("");
   });
 });
 
@@ -746,7 +770,7 @@ describe("document context", () => {
 
     expect(session.snapshotForTesting).toMatchObject({
       context: { dialect: "postgres", engine: "warehouse" },
-      text: "SELECT 2",
+      source: { originalText: "SELECT 2" },
     });
 
     const snapshot = session.snapshotForTesting;
@@ -768,7 +792,7 @@ describe("document context", () => {
       baseRevision: session.revision,
       context: { dialect: "postgres", engine: "warehouse" },
     });
-    expect(session.snapshotForTesting.text).toBe("SELECT 1");
+    expect(session.snapshotForTesting.source.originalText).toBe("SELECT 1");
     expect(session.snapshotForTesting.context.dialect).toBe("postgres");
   });
 
@@ -821,7 +845,7 @@ describe("document context", () => {
       } as never);
     });
     expect(session.revision).toBe(revision);
-    expect(session.snapshotForTesting.text).toBe("SELECT 1");
+    expect(session.snapshotForTesting.source.originalText).toBe("SELECT 1");
   });
 
   it("rejects accessors without invoking them", () => {