fix(incremental): round-2 review — terminator mention gate, defense restoration, checkpoint slimming

Second adversarial pass (4 finder angles, ~40 probes) over the previous
fix commit. One shipped-input correctness finding, rest defense-in-depth:

- terminator label mentions: the injection terminator is a synthetic
  DEFINITION, so a tail-side `[__aimd_injection_terminator__]` mention
  (shortcut/full/image form) resolved against it while the full parse
  renders literal text (probe-confirmed by two independent reviewers,
  persisting into the settled document). Structural fix over a doc
  caveat: frames whose pre-injection tail contains the label take the
  full path (tailMentionsTerminator gate); prefix mentions need no check
  (unresolved => taint-pinned into the tail; resolved => the real def
  wins first-def-wins over the terminator). Arbiter fixtures for all
  three reference forms.
- resume-validity now gates the sticky-uninjectable short-circuit too (a
  regressed cache boundary must degrade to a fresh walk, never a stale
  verdict), and a position-less top-level mdast child makes the plan
  uncacheable + restarts fresh (a resumed walk would re-visit it and
  duplicate its cached events — unreachable with the shipped chain,
  closed structurally).
- G4's straddle scan and the prefix cut give back their round-1 early
  breaks: the defensive gate must not share the ordered-children
  assumption it exists to catch (the O(top-level) cost is noise); the
  injection walk keeps its breaks (event ORDER already assumes document
  order there).
- alignPrefixCut's leading-run branch tightened to the probe-verified
  model exactly: excess leading text is only accepted with ZERO stripped
  slots (hoist merges into a slot whenever one exists).
- MarkdownContent wraps the engine in a throw fence: the scan checkpoint
  is advanced in place BEFORE the tail parse, so a mid-frame throw (an
  engine bug, or a plugin choking on the synthetic tail shape) used to
  leave the ref holding a state whose checkpoint described newer content
  — with a tail-repair preprocessor's non-append shapes that could
  eventually freeze an unsafe prefix. Now: clear the ref, full-parse the
  frame, dev-mode console.error.
- SSR routes through the plain full-parse branch: the engine's scan only
  exists to seed the NEXT frame's checkpoint, and a per-request render
  has no next frame (was a dead O(document) line-lex per SSR'd message).
- FreezeScanCheckpoint drops its lines array — it retained a full copy
  of the document (~2-3x doc size per mounted instance) to answer a
  question only ever asked about ONE line: blocker 4's next-line settle.
  Candidates now settle eagerly (tri-state on the candidate; only the
  newest can be pending), same verdicts, resume≡fresh property intact.
- incrementalParseEnabled JSDoc updated to the v2 gate list (the two
  v1-only bullets — cross-chunk and [^ — were the IDE-visible surface);
  stale 'caller bypasses' test comment fixed.

Cleared by probe in this round (no action): StrictMode double-invoke /
concurrent abandonment / checkpoint resume idempotence; rolling-line and
countNewlines bound fenceposts vs reference implementations; plan-cache
monotonicity under splice/full alternation and suffix churn; every
terminator/leading-hoist attack shape (setext, lazy continuation, defList
after link-def-last, 1-3-space indents, double tables, growing hoist,
comment+table starts, phantom-suffix interplay); all round-1 deletions
verified consumer-free with equivalent replacement coverage.

810 tests green (unit + browser); typecheck/lint/prettier/build clean.

Author: AIEPhoenix

packages/core/src/components/MarkdownContent.tsx

@@ -463,7 +463,13 @@ const BlockMemoizedRenderer = memo(
       // plus the replay-regenerated footer (hast). When the flag is off, the
       // state is CLEARED — a later eligible frame must never splice against
       // trees parsed under different conditions.
-      if (!config.incrementalParseEnabled) {
+      //
+      // SSR takes this branch too: the engine's first-frame scan exists to
+      // seed the NEXT frame's checkpoint, and a per-request server render
+      // has no next frame — routing through the engine would pay a dead
+      // O(document) line-lex per request. Hydration is unaffected (the
+      // client's first frame rebuilds from null either way).
+      if (!config.incrementalParseEnabled || typeof window === 'undefined') {
         incrementalStateRef.current = null;
         // Dev-only stage telemetry (`ai-markdown:stage:*` performance
         // measures; no-op in production). Wraps only the stage calls — the
@@ -480,23 +486,47 @@ const BlockMemoizedRenderer = memo(
         return { mdast: parsed.mdast, hast: hastRoot };
       }
 
-      const result = advanceIncrementalParse(incrementalStateRef.current, content ?? '', {
-        remarkPlugins,
-        rehypePlugins,
-        remarkRehypeOptions: mergedRemarkRehypeOptions,
-        // Identity tuple over every parse input beyond the content itself.
-        // Deliberately covers MORE than the G3 flush's 12 fields (handlers /
-        // preserveForBodyHarvest / documentId can change without touching
-        // any G3 field — e.g. a `preserveOrphanReferences` flip). The
-        // phantom label sets are deliberately NOT here: their churn tracks
-        // the suffix (always re-parsed with the tail), never the prefix.
-        depsKey: [remarkPlugins, rehypePlugins, remarkRehypeOptions, handlers, preserveForBodyHarvest, documentId],
-        defListEnabled: config.extraSyntaxSupported.includes(AIMarkdownRenderExtraSyntax.DEFINITION_LIST),
-        phantomSuffix,
-        measure: measureHere,
-      });
-      incrementalStateRef.current = result.nextState;
-      return { mdast: result.mdast, hast: result.hast };
+      try {
+        const result = advanceIncrementalParse(incrementalStateRef.current, content ?? '', {
+          remarkPlugins,
+          rehypePlugins,
+          remarkRehypeOptions: mergedRemarkRehypeOptions,
+          // Identity tuple over every parse input beyond the content itself.
+          // Deliberately covers MORE than the G3 flush's 12 fields (handlers /
+          // preserveForBodyHarvest / documentId can change without touching
+          // any G3 field — e.g. a `preserveOrphanReferences` flip). The
+          // phantom label sets are deliberately NOT here: their churn tracks
+          // the suffix (always re-parsed with the tail), never the prefix.
+          depsKey: [remarkPlugins, rehypePlugins, remarkRehypeOptions, handlers, preserveForBodyHarvest, documentId],
+          defListEnabled: config.extraSyntaxSupported.includes(AIMarkdownRenderExtraSyntax.DEFINITION_LIST),
+          phantomSuffix,
+          measure: measureHere,
+        });
+        incrementalStateRef.current = result.nextState;
+        return { mdast: result.mdast, hast: result.hast };
+      } catch (error) {
+        // The engine mutates prev's scan checkpoint IN PLACE before the tail
+        // parse/splice — a throw mid-frame (an engine bug, or a plugin
+        // choking on the synthetic tail source) leaves the retained state's
+        // checkpoint describing content the state's trees do not. Clearing
+        // the ref restores the "state is CLEARED when unusable" discipline;
+        // the frame then renders via the ordinary full pipeline so one bad
+        // frame cannot take the surface down.
+        incrementalStateRef.current = null;
+        if (process.env.NODE_ENV !== 'production') {
+          console.error('[ai-react-markdown] incremental parse failed — full parse fallback for this frame:', error);
+        }
+        const parsed = measureHere('parse', () =>
+          parseStage({
+            children: augmented,
+            remarkPlugins,
+            rehypePlugins,
+            remarkRehypeOptions: mergedRemarkRehypeOptions,
+          })
+        );
+        const hastRoot = measureHere('transform', () => transformStage(parsed));
+        return { mdast: parsed.mdast, hast: hastRoot };
+      }
     }, [
       content,
       targetPhantoms,

packages/core/src/components/incrementalParse/advanceIncrementalParse.ts

@@ -49,7 +49,13 @@ import type { Root as MdastRoot } from 'mdast';
 
 import { parseStage, transformStage, type Options as MarkdownOptions } from '../markdown';
 import { computeFreezeBoundary, type FreezeScanCheckpoint } from './computeFreezeBoundary';
-import { buildInjectionPrefix, collectPrefixInjection, spliceTrees, type CachedInjectionPlan } from './spliceParse';
+import {
+  buildInjectionPrefix,
+  collectPrefixInjection,
+  spliceTrees,
+  tailMentionsTerminator,
+  type CachedInjectionPlan,
+} from './spliceParse';
 
 export interface IncrementalParseState {
   /** The CHUNK's own text — excludes the phantom suffix. */
@@ -195,24 +201,34 @@ export function advanceIncrementalParse(
   // G3
   const boundary = Math.min(freshBoundary, prev!.stableBoundary);
   if (boundary <= 0) return fullPath();
-  // G4 (defensive) — children are position-ordered, so the walk ends at the
-  // first child starting at/past the boundary.
+  // G4 (defensive) — deliberately a FULL scan, no ordered-children early
+  // break: this is the gate that catches ordering/straddle violations, so
+  // it must not share the assumption it defends against (round-2 review).
+  // O(top-level children) per frame is noise next to the tail parse.
   for (const child of prev!.mdast.children) {
     const start = child.position?.start?.offset;
     const end = child.position?.end?.offset;
-    if (start !== undefined && start >= boundary) break;
-    if (start !== undefined && end !== undefined && end > boundary) {
+    if (start !== undefined && end !== undefined && start < boundary && end > boundary) {
       return fullPath();
     }
   }
 
+  // The one input class where the injection's synthetic terminator label
+  // could change how the tail parses: the tail literally mentioning it.
+  // Checked on the PRE-injection tail (the injection itself contains the
+  // label). Pathological by construction — correctness over splice rate.
+  const tailAndSuffix = content.slice(boundary) + phantomSuffix;
+  if (tailMentionsTerminator(tailAndSuffix)) return fullPath();
+
   const plan = collectPrefixInjection(prev!.mdast, prev!.content, boundary, injectionPlan);
-  injectionPlan = { boundary, events: plan.events, uninjectable: plan.uninjectable };
+  if (plan.cacheable !== false) {
+    injectionPlan = { boundary, events: plan.events, uninjectable: plan.uninjectable };
+  }
   // A nested definition/footnote-def that cannot be re-injected verbatim —
   // take the full path this frame rather than splice without it (A3).
   if (plan.uninjectable) return fullPath();
   const injection = buildInjectionPrefix(plan.events);
-  const tailSource = injection.text + content.slice(boundary) + phantomSuffix;
+  const tailSource = injection.text + tailAndSuffix;
   const tail = runPipeline(tailSource, options);
   const spliced = spliceTrees({
     prevMdast: prev!.mdast,

packages/core/src/components/incrementalParse/computeFreezeBoundary.test.ts

@@ -112,7 +112,7 @@ describe('computeFreezeBoundary — continuation blockers', () => {
     expect(computeFreezeBoundary(terminated, OFF)).toBe(terminated.indexOf('tail'));
   });
 
-  test('footnote definition context blocks (defensive; caller bypasses [^ anyway)', () => {
+  test('footnote definition context blocks (LOAD-BEARING since v2: defs splice via replay)', () => {
     expect(computeFreezeBoundary('[^n]: body\n\n', OFF)).toBe(0);
   });
 });

packages/core/src/components/incrementalParse/computeFreezeBoundary.ts

@@ -115,8 +115,12 @@ interface Candidate {
   htmlBalanced: boolean;
   /** Rolling continuation-hazard verdict at emission (blocker 3). */
   hazard: boolean;
-  /** Index into the checkpoint's confirmed-lines array. */
-  lineIndex: number;
+  /** Blocker-4 settle verdict, decided by the NEXT confirmed line (`null`
+   *  while that line hasn't confirmed — only the newest candidate can be
+   *  pending). Storing the verdict instead of the line lets the checkpoint
+   *  drop its lines array, which retained a full copy of the document
+   *  (round-2 review: ~2-3× doc size per mounted instance). */
+  defListSettled: boolean | null;
 }
 
 interface UnresolvedRef {
@@ -131,8 +135,6 @@ export interface FreezeScanCheckpoint {
   defListEnabled: boolean;
   /** Start offset of the first line NOT yet baked into this checkpoint. */
   confirmedOffset: number;
-  /** 1-based count guard: text.length the checkpoint was last advanced on. */
-  lines: LineRec[];
   candidates: Candidate[];
   defs: Map<string, number>; // normalized label → def line end offset
   footnoteDefs: Map<string, number>;
@@ -279,7 +281,6 @@ function freshCheckpoint(defListEnabled: boolean): FreezeScanCheckpoint {
   return {
     defListEnabled,
     confirmedOffset: 0,
-    lines: [],
     candidates: [],
     defs: new Map(),
     footnoteDefs: new Map(),
@@ -365,14 +366,12 @@ export function computeFreezeBoundary(
   let earliestUnresolved = Infinity;
   for (const ref of cp.unresolvedRefs) earliestUnresolved = Math.min(earliestUnresolved, ref.offset);
 
-  // ── blocker 4: defList settled check (looks FORWARD from a candidate) ──
+  // ── blocker 4: defList settled check (decided by the NEXT line) ──
   const defListSettled = (c: Candidate): boolean => {
     if (!options.defListEnabled || c.blankRun >= 2) return true;
-    for (let i = c.lineIndex + 1; i < cp.lines.length; i++) {
-      const ln = cp.lines[i];
-      if (ln.blank) return true; // a second blank makes the run ≥ 2 — the backward scan cannot cross it
-      return !canBecomeDdLine(ln.text, true);
-    }
+    // Confirmed next lines settle candidates eagerly in processConfirmedLine;
+    // only the newest candidate can still be pending here.
+    if (c.defListSettled !== null) return c.defListSettled;
     if (tailLine) return !canBecomeDdLine(tailLine.text, false);
     return false; // no next line yet — a future `: desc` could still claim the block above
   };
@@ -393,7 +392,13 @@ export function computeFreezeBoundary(
 
 /** Bake one confirmed line into the checkpoint. */
 function processConfirmedLine(cp: FreezeScanCheckpoint, ln: LineRec, text: string): void {
-  cp.lines.push(ln);
+  // Blocker-4 eager settle: this line is the "next confirmed line" of the
+  // newest candidate. The verdict uses the RAW line exactly like the old
+  // lines-array lookback did (fence/math state deliberately not consulted).
+  const newest = cp.candidates[cp.candidates.length - 1];
+  if (newest && newest.defListSettled === null) {
+    newest.defListSettled = ln.blank ? true : !canBecomeDdLine(ln.text, true);
+  }
   const isBlockStart = cp.prevLineBlank;
   const applyTag = (tag: string, closing: boolean): void => {
     if (closing) {
@@ -484,7 +489,7 @@ function processConfirmedLine(cp: FreezeScanCheckpoint, ln: LineRec, text: strin
       blankRun: cp.blankRun,
       htmlBalanced: cp.openTotal === 0 && !cp.commentOpen && !cp.piOpen && !cp.declOpen && !cp.cdataOpen,
       hazard: cp.hazardVerdict,
-      lineIndex: cp.lines.length - 1,
+      defListSettled: null,
     });
     cp.paragraphHasUnpairedRun = false;
     cp.prevLineBlank = true;

packages/core/src/components/incrementalParse/spliceEquivalence.test.ts

@@ -548,6 +548,23 @@ describe('splice equivalence — footnote injection replay', () => {
     }
   });
 
+  test('tail mentioning the terminator label falls back instead of mis-resolving (round-2)', () => {
+    // The injection's terminator is a synthetic DEFINITION — a tail-side
+    // `[label]` mention would resolve against it while the full parse
+    // renders literal text. The engine must take the full path for such
+    // frames (pathological input; correctness over splice rate). The
+    // shortcut-, full-, and image-reference forms are all covered.
+    const mentions = [
+      'see [__aimd_injection_terminator__] maybe.\n',
+      'see [text][__aimd_injection_terminator__] maybe.\n',
+      'see ![alt][__aimd_injection_terminator__] maybe.\n',
+    ];
+    for (const mention of mentions) {
+      const frame0 = 'note[^q] first.\n\n[^q]: def body\n\nplain settles.\n\n';
+      assertStreamEquivalence('terminator-mention', [frame0, frame0 + mention], BASELINE);
+    }
+  });
+
   test('injection continuation leaks (final-review R1): defList `: desc` claim through the join', () => {
     // The original document separates with TWO blank lines (blankRun>=2 is
     // defList-claim-immune), but the injection joins to the tail with ONE —

packages/core/src/components/incrementalParse/spliceParse.ts

@@ -63,6 +63,11 @@ export interface PrefixInjectionPlan {
   /** True when an event exists that CANNOT be injected reliably — the caller
    *  must fall back to a full parse for this frame (safe, one-frame cost). */
   uninjectable: boolean;
+  /** False when the plan must NOT be cached for resume: a position-less
+   *  top-level child cannot be partitioned by the resume offset, so a
+   *  resumed walk would re-visit it and duplicate its events (round-2
+   *  review). Fresh walks stay correct — they just can't be incremental. */
+  cacheable?: boolean;
 }
 
 /** A plan cached in engine state: valid for any LATER boundary of the same
@@ -103,12 +108,18 @@ export function collectPrefixInjection(
   // frame to frame, so a cached plan only needs the children in
   // [resume.boundary, boundary) appended — without this the deep visit
   // re-walks the ENTIRE prefix every splice frame (the collectDefLabels
-  // O(stream²) shape). `uninjectable` is sticky: the offending node stays
-  // in the prefix for the lineage's lifetime.
-  if (resume && resume.uninjectable) return { events: resume.events, uninjectable: true };
-  const resumeAt = resume && resume.boundary <= boundary ? resume.boundary : 0;
+  // O(stream²) shape). Boundary validity gates EVERYTHING resumed —
+  // including the sticky-uninjectable short-circuit: the boundary is
+  // monotone within an append lineage today, but a regressed cache must
+  // degrade to a fresh walk, never to a stale verdict (round-2 review).
+  const resumeValid = resume != null && resume.boundary <= boundary;
+  // `uninjectable` is sticky: the offending node stays in the prefix for
+  // the lineage's lifetime.
+  if (resumeValid && resume!.uninjectable) return { events: resume!.events, uninjectable: true };
+  const resumeAt = resumeValid ? resume!.boundary : 0;
   const events: InjectionEvent[] = resumeAt > 0 ? cloneEventsForAppend(resume!.events) : [];
   let uninjectable = false;
+  let cacheable = true;
   const pushRef = (token: string): void => {
     const last = events[events.length - 1];
     if (last && last.kind === 'refs') last.tokens.push(token);
@@ -163,11 +174,22 @@ export function collectPrefixInjection(
     // boundary can contribute a prefix event (E5), and nothing before the
     // resume point needs re-visiting.
     const start = child.position?.start?.offset;
-    if (start !== undefined && start >= boundary) break;
-    if (start !== undefined && start < resumeAt) continue;
+    if (start === undefined) {
+      // A position-less top-level child cannot be partitioned by offset: a
+      // resumed walk would re-visit it and DUPLICATE its cached events.
+      // Restart fresh once (correct by construction) and mark the plan
+      // uncacheable. Unreachable with the shipped chain (the parser
+      // positions every top-level node); defense for plugin-shaped trees.
+      if (resumeAt > 0) return collectPrefixInjection(mdast, content, boundary, null);
+      cacheable = false;
+      visit(child, false);
+      continue;
+    }
+    if (start >= boundary) break;
+    if (start < resumeAt) continue;
     visit(child, false);
   }
-  return { events, uninjectable };
+  return { events, uninjectable, cacheable };
 }
 
 /** Cached plans are shared with the previous state — clone the array, and
@@ -200,10 +222,25 @@ export interface InjectionPrefix {
  *  A column-0 link definition neutralizes both: it terminates a footnote
  *  body (not >=4-indented), cannot be claimed as a <dt> (not a paragraph),
  *  emits zero hast and zero wrap-separator slots, and is stripped with the
- *  rest of the injected region. The sentinel label collides with real
- *  content only if a document defines the exact same label — the same
- *  accepted-risk class as the phantom sentinel URLs. */
-const INJECTION_TERMINATOR = '[__aimd_injection_terminator__]: __aimd_sentinel_link__';
+ *  rest of the injected region.
+ *
+ *  Unlike the phantom sentinel URLs (values — they can never change parse
+ *  SHAPE), a definition LABEL is resolvable by any `[label]` mention, so a
+ *  document that literally writes the sentinel label would have its tail
+ *  mentions resolve against the terminator (round-2 review, probe-
+ *  confirmed). {@link tailMentionsTerminator} closes that structurally:
+ *  such frames take the full path instead of splicing. */
+const TERMINATOR_LABEL = '__aimd_injection_terminator__';
+const INJECTION_TERMINATOR = `[${TERMINATOR_LABEL}]: __aimd_sentinel_link__`;
+
+/** True when the tail input could REFERENCE the terminator's label — the
+ *  one input class where the splice's synthetic definition would change how
+ *  the tail parses. Prefix mentions need no check: an unresolved mention is
+ *  reference-tainted (pinned into the tail), and a resolved one has a real
+ *  def that wins first-def-wins over the terminator. */
+export function tailMentionsTerminator(tailSource: string): boolean {
+  return tailSource.includes(`[${TERMINATOR_LABEL}`);
+}
 
 /** Build the injection text prepended to the tail source (with per-footnote-
  *  def coordinate segments). Blocks are '\n\n'-joined: a def line directly
@@ -342,12 +379,13 @@ export function spliceTrees(input: SpliceInput): { mdast: MdastRoot; hast: HastR
   const lineDelta = prefixLines - injectedLines;
 
   // --- prefix cuts (non-mutating reads of the previous frame's trees) ---
+  // Filter semantics (no ordered-children early break): together with G4's
+  // full straddle scan this keeps the cut correct-or-bailing even for
+  // disordered trees a plugin-shaped mdast could present (round-2 review).
   const prefixMdast: MdastContent[] = [];
   for (const child of prevMdast.children) {
-    // Position-ordered — the first child at/past the boundary ends the prefix.
     const start = child.position?.start?.offset;
-    if (start !== undefined && start >= boundary) break;
-    if (start !== undefined) prefixMdast.push(child);
+    if (start !== undefined && start < boundary) prefixMdast.push(child);
   }
   const attrs = attributeHastChildren(prevMdast, prevHast, boundary);
   const cutRegion: HastContent[] = [];
@@ -550,11 +588,13 @@ function alignPrefixCut(
         }
       }
       if (nextIdx === -1) return null;
-      // j slots belong to the j leading stripped children; the rest is hoist.
-      // Only the no-stripped shape is probe-verified (hoist merges into the
-      // last slot when slots exist) — mixed shapes beyond one hoist text are
-      // out of model.
-      if (sepBuffer.length - nextIdx > 1) return null;
+      // Excess leading texts beyond the stripped-child slots can only be
+      // hoist, and hoist exists as a SEPARATE node only when there is no
+      // slot to merge into (nextIdx === 0, probe-verified). Any excess in
+      // the presence of slots is out of model → bail (round-2 review
+      // tightened this from `excess > 1`).
+      const excess = sepBuffer.length - nextIdx;
+      if (excess > (nextIdx === 0 ? 1 : 0)) return null;
     } else {
       const stripped = sawContent ? sepBuffer.length - 1 : sepBuffer.length;
       if (stripped < 0) return null;