refactor(datagrid): migrate callers to TableRows + delete legacy stack (#931)

* refactor(datagrid): introduce TableRowsStore alongside RowDataStore

* refactor(datagrid): introduce TableRowsController to drive NSTableView from Delta

* refactor(datagrid): mirror result delivery writes into TableRowsStore

* refactor(datagrid): switch JSON view and export to read TableRows

* refactor(datagrid): switch sidebar reads to TableRows

* refactor(datagrid): switch NSTableView delegate reads to TableRows

* refactor(datagrid): route cell edits through TableRows.edit and Delta

* refactor(datagrid): RowOperationsManager mutates TableRows and returns Delta

* refactor(datagrid): undo replay returns Delta and supports non-tail insert

* refactor(datagrid): sort moves to controller keyed by Row.id

* refactor(datagrid): display cache moves to coordinator keyed by Row.id

* refactor(datagrid): replace RowBuffer with TableRows; delete legacy row stack

* test(datagrid): update fixtures for RowDeltaApplying.applyDelta and StatusBarSnapshot rename

* fix(favorites): qualify ORDER BY columns in FTS-joined search query

* fix(datagrid): guard commitCellEdit against re-entry from reload-driven resignFirstResponder

* fix(datagrid): dispatch rowsInserted delta from loadMoreRows and read offset inside mutation

* fix(datagrid): dispatch fullReplace delta from fetchAllRows

* fix(datagrid): dispatch cellChanged delta from updateCellInTab

* fix(datagrid): remove TableRows entry on individual tab close

* refactor(datagrid): rebuildColumnMetadataCache reads live TableRows

* style(datagrid): drop doc comments per no-comments rule

* fix(datagrid): sync ResultSet snapshot with store on every mutation

Load More appended rows to the store but ResultSet.tableRows stayed
frozen at applyPhase1Result time. The data grid read through the active
ResultSet, so newly fetched rows showed empty cells.

Route every TableRows mutation through mutateActiveTableRows on
MainContentCoordinator. The helper updates the store, then writes the
new value back into the active ResultSet. Result-set switches go
through switchActiveResultSet so the store tracks the new active
snapshot. Read paths simplify to reading the store directly.

* fix(datagrid): unwrap optional result from addNewRow/duplicateRow

* fix(datagrid): clear modified cell highlight on undo, keep FK metadata across reloads

Two related visual regressions:

1. Edit cell -> yellow modified background -> undo -> value reverts but
   highlight persists. applyDataUndo updated pending state but did not
   bump reloadVersion, so the visual state cache was gated and the
   stale modifiedColumns set survived. Bumping reloadVersion forces a
   rebuild on the next render.

2. FK column arrow and dropdown chevron toggle visible/hidden on each
   reload of a table tab. applyPhase1Result rebuilt TableRows from
   scratch and only populated columnDefaults / columnForeignKeys /
   columnNullable / columnEnumValues when a fresh schema fetch ran.
   When isMetadataCached returned true (because metadata was in the
   previous TableRows), no fetch ran and the new TableRows wiped the
   metadata, which then caused the next reload to refetch -- so every
   other reload had FK info and every other one didn't. Carry the
   existing metadata over when the schema fetch is skipped.

* perf(datagrid): stop syncing ResultSet snapshot on every mutation

Inserting or undoing a row pegged the CPU at 100%. Each mutation went
through mutateActiveTableRows, which wrote the live TableRows back into
the active ResultSet's @observable tableRows property. That triggered a
full SwiftUI re-render of MainEditorContentView (which reads
rs.resultColumns / rs.errorMessage / etc), on top of the existing
observation triggers from changeManager.reloadVersion and
tabManager.tabs. Fast key-repeat undo or paste cascaded re-renders.

Move the per-ResultSet snapshot into a save-on-switch model:
mutateActiveTableRows now only writes the store, and
switchActiveResultSet saves the outgoing snapshot then loads the
incoming one. Edits in a pinned result set still survive switching
back, but routine inserts / undos / cell edits no longer cross the
@observable boundary on the ResultSet.

Also short-circuit the inserted-rows scan in
DataGridCoordinator.rebuildVisualStateCache: when the grid is unsorted,
read changeManager.insertedRowIndices directly instead of iterating
every row in TableRows. The full scan only runs in the sorted case
where display indices differ from storage indices.

* test(datagrid): fix EvictionTests — selected tab is intentionally not evicted

`evictInactiveRowData` deliberately skips the currently selected tab
("kept in memory so the user sees no refresh flicker"), but the migrated
tests added a tab and immediately called eviction — the new tab was the
selected tab, so eviction was a no-op and the assertions failed.

Fix: add a second tab so the first becomes background, then assert
eviction on the background tab.

* perf(datagrid): updateCache reads live tableRowsProvider so post-Delta count is fresh

* refactor(datagrid): replace editingCell binding with direct beginEditing call

addNewRow / duplicateSelectedRow now call coordinator.beginEditing(displayRow:column:) synchronously after applyDelta. The editingCell SwiftUI binding plumbing across 7 files is removed since no caller sets it to non-nil anymore. Each row-add press fully completes (commit prior edit, mutate model, apply delta, focus new cell) before the next press fires, so rapid Cmd+Shift+N keeps focus on the latest appended row instead of getting trapped in queued Tasks.

* chore: drop DATAGRID_REFACTOR.md handoff doc

* fix(datagrid): clear cell display cache when row data is replaced

setActiveTableRows now dispatches applyFullReplace to the active grid after every full row swap, routing through a single mutation surface instead of seven direct tableRowsStore.setTableRows callers across navigation, query helpers, multi-statement, FK navigation, and sidebar actions. Without the dispatch, the coordinator's RowID-keyed displayCache survived table switches and returned the previous table's formatted cell values for matching RowIDs, even though the cell views themselves had rebuilt with the new column set.

* refactor(datagrid): rename RowDeltaApplying to TableViewCoordinating, add dispatch regression tests

The protocol now exposes commitActiveCellEdit and beginEditing alongside the row-delta methods, so its name no longer matches its scope. Renaming to TableViewCoordinating tracks the conforming class TableViewCoordinator and the field DataTabGridDelegate.tableViewCoordinator.

TableRowsMutationTests verifies that setActiveTableRows dispatches applyFullReplace exactly once for the active tab and skips background tabs, locking in the displayCache invalidation contract that was missing before commit 0e967c2.

Author: datlechin

CHANGELOG.md

@@ -22,7 +22,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
-- Introduced TableRows, Row, and Delta value types in TablePro/Models/Query/ as the foundation for the data grid row model rewrite. No callers migrated yet (Phase C.1 of the DataGrid refactor).
+- Replaced RowBuffer / InMemoryRowProvider / RowDataStore with TableRows / TableRowsStore / TableRowsController. Mutations emit Delta events; the controller drives NSTableView via insertRows / removeRows / reloadData(forRowIndexes:). Sort and the display cache moved off the row provider into the data grid coordinator, keyed by Row.id.
+- Routed every TableRows mutation through `mutateActiveTableRows` on MainContentCoordinator so the active ResultSet's snapshot stays in sync with the store. The snapshot now refreshes only when the user switches result sets (saving the outgoing tab, loading the incoming one), so each insert / undo / paste no longer triggers an `@Observable` re-render of the whole editor. Fixes empty cells on Load More and CPU spikes when adding or undoing rows.
+- Undo of a cell edit clears the modified-cell highlight: `DataChangeManager.applyDataUndo` now bumps `reloadVersion` so the data grid rebuilds its visual state cache.
+- Reloading a table tab keeps cached column metadata (defaults, foreign keys, nullability, enum values) when no fresh schema fetch was needed, so the FK arrow and dropdown chevron stay visible across reloads instead of toggling.
 - DataChangeManager extracted a PendingChanges value type that owns cross-collection invariants for cell edits, row insertions, and deletions. DataChangeManager kept undo/redo registration, plugin SQL generation, and the `@Observable` boundary, dropping from ~960 to ~190 lines. The serialization DTO `TabPendingChanges` is renamed to `TabChangeSnapshot` to distinguish it from the live tracker.
 - AnyChangeManager uses ChangeManaging protocol instead of closure-based type erasure, removing all runtime `[Any]` downcasts
 - Row selection state moved from MainContentView @State to GridSelectionState @Observable class, preventing full view tree invalidation on every row click

TablePro/Core/ChangeTracking/AnyChangeManager.swift

@@ -7,6 +7,7 @@ protocol ChangeManaging: AnyObject {
     var reloadVersion: Int { get }
     var canRedo: Bool { get }
     var rowChanges: [RowChange] { get }
+    var insertedRowIndices: Set<Int> { get }
     func isRowDeleted(_ rowIndex: Int) -> Bool
     func recordCellChange(
         rowIndex: Int,
@@ -30,6 +31,7 @@ final class AnyChangeManager {
     var reloadVersion: Int { wrapped.reloadVersion }
     var canRedo: Bool { wrapped.canRedo }
     var rowChanges: [RowChange] { wrapped.rowChanges }
+    var insertedRowIndices: Set<Int> { wrapped.insertedRowIndices }
 
     func isRowDeleted(_ rowIndex: Int) -> Bool {
         wrapped.isRowDeleted(rowIndex)

TablePro/Core/ChangeTracking/DataChangeManager.swift

@@ -17,6 +17,21 @@ struct UndoResult {
     let needsRowRemoval: Bool
     let needsRowRestore: Bool
     let restoreRow: [String?]?
+    let delta: Delta
+
+    init(
+        action: UndoAction,
+        needsRowRemoval: Bool,
+        needsRowRestore: Bool,
+        restoreRow: [String?]?,
+        delta: Delta = .none
+    ) {
+        self.action = action
+        self.needsRowRemoval = needsRowRemoval
+        self.needsRowRestore = needsRowRestore
+        self.restoreRow = restoreRow
+        self.delta = delta
+    }
 }
 
 /// Manager for tracking and applying data changes
@@ -259,7 +274,10 @@ final class DataChangeManager: ChangeManaging {
                 originalDBValue: newValue, newValue: previousValue, originalRow: originalRow
             )
         }
-        lastUndoResult = UndoResult(action: action, needsRowRemoval: false, needsRowRestore: false, restoreRow: nil)
+        lastUndoResult = UndoResult(
+            action: action, needsRowRemoval: false, needsRowRestore: false, restoreRow: nil,
+            delta: .cellChanged(row: rowIndex, column: columnIndex)
+        )
     }
 
     private func applyRowInsertionUndo(rowIndex: Int, action: UndoAction) {
@@ -274,12 +292,14 @@ final class DataChangeManager: ChangeManaging {
         if pending.isRowInserted(rowIndex) {
             _ = pending.undoRowInsertion(rowIndex: rowIndex)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil
+                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil,
+                delta: .rowsRemoved(IndexSet(integer: rowIndex))
             )
         } else {
             pending.reinsertRow(rowIndex: rowIndex, columns: columns, savedValues: savedValues)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: savedValues
+                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: savedValues,
+                delta: .rowsInserted(IndexSet(integer: rowIndex))
             )
         }
     }
@@ -292,12 +312,14 @@ final class DataChangeManager: ChangeManaging {
         if pending.isRowDeleted(rowIndex) {
             _ = pending.undoRowDeletion(rowIndex: rowIndex)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: originalRow
+                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: originalRow,
+                delta: .fullReplace
             )
         } else {
             pending.reapplyRowDeletion(rowIndex: rowIndex, originalRow: originalRow)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil
+                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil,
+                delta: .fullReplace
             )
         }
     }
@@ -315,14 +337,16 @@ final class DataChangeManager: ChangeManaging {
                 _ = pending.undoRowDeletion(rowIndex: rowIndex)
             }
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: nil
+                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: nil,
+                delta: .fullReplace
             )
         } else {
             for (rowIndex, originalRow) in rows {
                 pending.reapplyRowDeletion(rowIndex: rowIndex, originalRow: originalRow)
             }
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil
+                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil,
+                delta: .fullReplace
             )
         }
     }
@@ -335,15 +359,18 @@ final class DataChangeManager: ChangeManaging {
         }
 
         let firstInserted = rowIndices.first.map { pending.isRowInserted($0) } ?? false
+        let indices = IndexSet(rowIndices)
         if firstInserted {
             _ = pending.undoBatchRowInsertion(rowIndices: rowIndices, columnCount: columns.count)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil
+                action: action, needsRowRemoval: true, needsRowRestore: false, restoreRow: nil,
+                delta: .rowsRemoved(indices)
             )
         } else {
             pending.reinsertBatch(rowIndices: rowIndices, rowValues: rowValues, columns: columns)
             lastUndoResult = UndoResult(
-                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: nil
+                action: action, needsRowRemoval: false, needsRowRestore: true, restoreRow: nil,
+                delta: .rowsInserted(indices)
             )
         }
     }

TablePro/Core/Plugins/QueryResultExportDataSource.swift

@@ -7,9 +7,6 @@ import Foundation
 import os
 import TableProPluginKit
 
-/// In-memory `PluginExportDataSource` backed by a RowBuffer snapshot.
-/// Allows export plugins (CSV, JSON, SQL, XLSX, MQL) to export query results
-/// without modification to the plugins themselves.
 final class QueryResultExportDataSource: PluginExportDataSource, @unchecked Sendable {
     let databaseTypeId: String
 
@@ -20,14 +17,12 @@ final class QueryResultExportDataSource: PluginExportDataSource, @unchecked Send
 
     private static let logger = Logger(subsystem: "com.TablePro", category: "QueryResultExportDataSource")
 
-    init(rowBuffer: RowBuffer, databaseType: DatabaseType, driver: DatabaseDriver?) {
+    init(tableRows: TableRows, databaseType: DatabaseType, driver: DatabaseDriver?) {
         self.databaseTypeId = databaseType.rawValue
         self.driver = driver
-
-        // Snapshot data at init time for thread safety
-        self.columns = rowBuffer.columns
-        self.columnTypeNames = rowBuffer.columnTypes.map { $0.rawType ?? "" }
-        self.rows = rowBuffer.rows
+        self.columns = tableRows.columns
+        self.columnTypeNames = tableRows.columnTypes.map { $0.rawType ?? "" }
+        self.rows = tableRows.rows.map(\.values)
     }
 
     func streamRows(table: String, databaseName: String) -> AsyncThrowingStream<PluginStreamElement, Error> {

TablePro/Core/Plugins/StreamingQueryExportDataSource.swift

@@ -4,7 +4,7 @@
 //
 //  Streaming export data source for query results.
 //  Re-executes the query and streams rows directly from the database to the export plugin,
-//  bypassing RowBuffer. Allows exporting large result sets without loading all rows into memory.
+//  bypassing in-memory storage. Allows exporting large result sets without loading all rows into memory.
 //
 
 import Foundation

TablePro/Core/SchemaTracking/StructureChangeManager.swift

@@ -887,6 +887,8 @@ final class StructureChangeManager: ChangeManaging {
 
     var rowChanges: [RowChange] { [] }
 
+    var insertedRowIndices: Set<Int> { [] }
+
     func isRowDeleted(_ rowIndex: Int) -> Bool { false }
 
     func recordCellChange(

TablePro/Core/Services/Export/ExportService.swift

@@ -181,15 +181,15 @@ final class ExportService {
     // MARK: - Query Results Export
 
     func exportQueryResults(
-        rowBuffer: RowBuffer,
+        tableRows: TableRows,
         config: ExportConfiguration,
         to url: URL
     ) async throws {
         guard let plugin = PluginManager.shared.exportPlugins[config.formatId] else {
             throw ExportError.formatNotFound(config.formatId)
         }
 
-        let totalRows = rowBuffer.rows.count
+        let totalRows = tableRows.count
         state = ExportState(isExporting: true, totalTables: 1, totalRows: totalRows)
         isCancelled = false
 
@@ -201,7 +201,7 @@ final class ExportService {
         }
 
         let dataSource = QueryResultExportDataSource(
-            rowBuffer: rowBuffer,
+            tableRows: tableRows,
             databaseType: databaseType,
             driver: driver
         )

TablePro/Core/Services/Formatting/CellDisplayFormatter.swift

@@ -3,7 +3,7 @@
 //  TablePro
 //
 //  Pure formatter that transforms raw cell values into display-ready strings.
-//  Used by InMemoryRowProvider's display cache to compute values once per cell.
+//  Used by the data grid coordinator's display cache to compute values once per cell.
 //
 
 import Foundation