fix(extract): return tables as paragraph-granular blocks (SD-2672) (#2925)

* fix(extract): return tables as paragraph-granular blocks (SD-2672)

doc.extract() was flattening tables into one joined string, which broke
RAG chunking and made table citations unreachable via scrollToElement.
Walk tables directly and emit one block per paragraph-like descendant
of each origin cell, tagged with tableContext so consumers can group
back to cell, row, or whole table.

- gridBefore/gridAfter placeholder cells are skipped via the
  __placeholder attr; they are layout artifacts with no user content.
- Block SDTs (structuredContentBlock) are transparent, so tables
  wrapped in content controls are not re-flattened through the
  wrapper's textContent.
- Cell paths use physical row-and-cell child indexes so deterministic
  fallback nodeIds agree with buildBlockIndex, keeping the
  scrollToElement round-trip stable for paragraphs that lack paraId
  and sdBlockId inside horizontally merged tables.

Tested: 13 behavior tests (7 existing SD-2525 + 6 new SD-2672),
5 new adapter unit tests, plus the full document-api-adapters suite
(3105 tests) and document-api bun suite (1362 tests).

* fix(extract): recurse through unrecognized block wrappers (SD-2672)

The new table walker only emitted blocks for recognized types and
silently dropped anything else, including their block children. That
regressed coverage versus the old textContent walk for `documentSection`,
`documentPartObject`, and `shapeContainer`, which all declare
block-level content but aren't in EMITTABLE_BLOCK_TYPES. Treat any
unrecognized block with block-level children as transparent and recurse
into it, so paragraphs nested inside these wrappers still surface with
their enclosing tableContext. Adds a unit test covering a
`documentSection` inside a table cell.

* test(extract): add DOCX-import-driven coverage for table edge cases (SD-2672)

The adapter unit tests hit the algorithm via schema-constructed PM docs,
which skips the importer entirely. This adds a second layer of tests
that load real Word-authored .docx files, run them through the full
import pipeline, and assert extract output. Closes the gap the code
review flagged for a customer-facing legal RAG contract.

Fixtures authored via Word COM + local OOXML patching:
- sd-2672-plain-3x3.docx: baseline table, no merges or placeholders
- sd-2672-merged-table.docx: colspan=2 and rowspan=2 anchors
- sd-2672-rtl-table.docx: bidiVisual RTL table
- sd-2672-gridbefore-vmerge.docx: w:gridBefore + w:vMerge=restart/continue
- sd-2672-sdt-table.docx: table wrapped in a w:sdt block (content control)
- sd-2672-nested-table.docx: 2x2 table inside cell (1,1) of outer table
- sd-2672-multipara-cell.docx: cell (0,0) with two paragraphs

The build-sd-2672-fixtures.mjs script regenerates the patched variants
from the Word-authored base, using JSZip + regex/XmlDocument surgery.

Tests assert: per-cell content lands at correct logical grid coords,
merged anchors carry rowspan/colspan, RTL tables still report columns
0..N-1, gridBefore placeholders don't emit phantom blocks, SDT wrappers
are transparent, nested tables get a fresh tableOrdinal with parent
coordinates, multi-paragraph cells emit one block per paragraph with
shared tableContext, and scrollToElement round-trips a merged-cell
paragraph nodeId.

* chore(tests): drop SD-2672 fixture build script

The script was added alongside the fixtures to regenerate the OOXML-patched
variants from a Word-authored base. It isn't carrying its weight: fixtures
are committed as static binaries, the regex-based XML patching is fragile
to Word COM output changes, and the commit history already documents how
each fixture was constructed. If we need a new edge-case fixture later,
hand-authoring it once is simpler than maintaining a generator.

* chore(tests): drop stale script reference in extract-docx error

Author: caio-pizzol

apps/docs/document-api/reference/_generated-manifest.json

@@ -1018,5 +1018,5 @@
     }
   ],
   "marker": "{/* GENERATED FILE: DO NOT EDIT. Regenerate via `pnpm run docapi:sync`. */}",
-  "sourceHash": "e74a36833ec8587b67447a79517de348cfc9b4bba1c564729c184f6d5464a018"
+  "sourceHash": "c8670fb494b56c19fbd09a7bada35974fbb3c22d938f6a5e01eee6e8467961c0"
 }

apps/docs/document-api/reference/extract.mdx

@@ -49,6 +49,15 @@ _No fields._
     {
       "headingLevel": 1,
       "nodeId": "node-def456",
+      "tableContext": {
+        "colspan": 1,
+        "columnIndex": 1,
+        "parentRowIndex": 1,
+        "parentTableOrdinal": 1,
+        "rowIndex": 1,
+        "rowspan": 1,
+        "tableOrdinal": 1
+      },
       "text": "Hello, world.",
       "type": "example"
     }
@@ -110,12 +119,57 @@ _No fields._
             "description": "Stable block ID — pass to scrollToElement() for navigation.",
             "type": "string"
           },
+          "tableContext": {
+            "additionalProperties": false,
+            "properties": {
+              "colspan": {
+                "description": "Number of columns the cell spans.",
+                "type": "integer"
+              },
+              "columnIndex": {
+                "description": "0-based logical grid column, not the row child order.",
+                "type": "integer"
+              },
+              "parentColumnIndex": {
+                "description": "Column index in the parent table. Set with parentTableOrdinal.",
+                "type": "integer"
+              },
+              "parentRowIndex": {
+                "description": "Row index in the parent table. Set with parentTableOrdinal.",
+                "type": "integer"
+              },
+              "parentTableOrdinal": {
+                "description": "Ordinal of the parent table when the containing table is nested.",
+                "type": "integer"
+              },
+              "rowIndex": {
+                "description": "0-based row index of the containing cell.",
+                "type": "integer"
+              },
+              "rowspan": {
+                "description": "Number of rows the cell spans.",
+                "type": "integer"
+              },
+              "tableOrdinal": {
+                "description": "0-based table ordinal, unique within one extract() result.",
+                "type": "integer"
+              }
+            },
+            "required": [
+              "tableOrdinal",
+              "rowIndex",
+              "columnIndex",
+              "rowspan",
+              "colspan"
+            ],
+            "type": "object"
+          },
           "text": {
             "description": "Full plain text content of the block.",
             "type": "string"
           },
           "type": {
-            "description": "Block type: paragraph, heading, listItem, table, image, etc.",
+            "description": "Block type: paragraph, heading, listItem, image, tableOfContents.",
             "type": "string"
           }
         },

packages/document-api/src/contract/schemas.ts

@@ -2963,9 +2963,40 @@ const operationSchemas: Record<OperationId, OperationSchemaSet> = {
           items: objectSchema(
             {
               nodeId: { type: 'string', description: 'Stable block ID — pass to scrollToElement() for navigation.' },
-              type: { type: 'string', description: 'Block type: paragraph, heading, listItem, table, image, etc.' },
+              type: {
+                type: 'string',
+                description: 'Block type: paragraph, heading, listItem, image, tableOfContents.',
+              },
               text: { type: 'string', description: 'Full plain text content of the block.' },
               headingLevel: { type: 'integer', description: 'Heading level (1–6). Only present for headings.' },
+              tableContext: objectSchema(
+                {
+                  tableOrdinal: {
+                    type: 'integer',
+                    description: '0-based table ordinal, unique within one extract() result.',
+                  },
+                  parentTableOrdinal: {
+                    type: 'integer',
+                    description: 'Ordinal of the parent table when the containing table is nested.',
+                  },
+                  parentRowIndex: {
+                    type: 'integer',
+                    description: 'Row index in the parent table. Set with parentTableOrdinal.',
+                  },
+                  parentColumnIndex: {
+                    type: 'integer',
+                    description: 'Column index in the parent table. Set with parentTableOrdinal.',
+                  },
+                  rowIndex: { type: 'integer', description: '0-based row index of the containing cell.' },
+                  columnIndex: {
+                    type: 'integer',
+                    description: '0-based logical grid column, not the row child order.',
+                  },
+                  rowspan: { type: 'integer', description: 'Number of rows the cell spans.' },
+                  colspan: { type: 'integer', description: 'Number of columns the cell spans.' },
+                },
+                ['tableOrdinal', 'rowIndex', 'columnIndex', 'rowspan', 'colspan'],
+              ),
             },
             ['nodeId', 'type', 'text'],
           ),

packages/document-api/src/types/extract.types.ts

@@ -4,15 +4,59 @@ import type { CommentStatus, TrackChangeType } from './index.js';
 // extract
 // ---------------------------------------------------------------------------
 
+/**
+ * Table coordinates for an {@link ExtractBlock} that lives inside a table cell.
+ *
+ * Blocks inside tables are extracted at paragraph granularity (one entry per
+ * paragraph/heading/listItem/image/sdt/tableOfContents in each cell). Group
+ * by these fields to reconstruct cells, rows, or whole tables:
+ *
+ * - cell:  group by `tableOrdinal + rowIndex + columnIndex`
+ * - row:   group by `tableOrdinal + rowIndex`
+ * - table: group by `tableOrdinal`
+ */
+export interface ExtractTableContext {
+  /** 0-based table ordinal, unique within one `extract()` result. */
+  tableOrdinal: number;
+  /** Ordinal of the parent table when this block is inside a nested table. */
+  parentTableOrdinal?: number;
+  /** Row index within the parent table. Only set with `parentTableOrdinal`. */
+  parentRowIndex?: number;
+  /** Column index within the parent table. Only set with `parentTableOrdinal`. */
+  parentColumnIndex?: number;
+  /** 0-based row index of the containing cell. */
+  rowIndex: number;
+  /** 0-based logical grid column of the containing cell, not the row's child order. */
+  columnIndex: number;
+  /** Number of rows the containing cell spans. 1 for unmerged cells. */
+  rowspan: number;
+  /** Number of columns the containing cell spans. 1 for unmerged cells. */
+  colspan: number;
+}
+
+/**
+ * One addressable unit of document content.
+ *
+ * Extraction is paragraph-granular: tables are NOT returned as a single block.
+ * Paragraph-like descendants of table cells are emitted individually with
+ * `tableContext` attached.
+ *
+ * Block SDTs (structured document tags / content controls) are transparent:
+ * their children emit individually as if they were direct children of the
+ * enclosing container. No wrapper `sdt` block is emitted. This prevents
+ * SDT-wrapped tables from re-flattening through the wrapper's textContent.
+ */
 export interface ExtractBlock {
-  /** Stable block ID — pass to `scrollToElement()` for navigation. */
+  /** Stable block ID. Pass to `scrollToElement()` for navigation. */
   nodeId: string;
-  /** Block type: paragraph, heading, listItem, table, image, etc. */
+  /** Block type: paragraph, heading, listItem, image, tableOfContents. */
   type: string;
   /** Full plain text content of the block. */
   text: string;
-  /** Heading level (1–6). Only present for headings. */
+  /** Heading level (1-6). Only present for headings. */
   headingLevel?: number;
+  /** Table coordinates. Only present for blocks inside a table cell. */
+  tableContext?: ExtractTableContext;
 }
 
 export interface ExtractComment {