Fix #1629: resolve memory leaks in LazilyTransformingAstService and UndoRedo (#1638)

## Summary

Fixes #1629. Closes #1633. Closes #1634.

Two unbounded memory leaks in long-running HyperFormula instances:

- **`LTAS.transformations[]`** grew linearly with every structural
operation (addRows, removeRows, moveCells, etc.) and was never cleaned
up. Fixed by introducing **threshold-based compaction** with
`versionOffset` — once 50+ transformations accumulate, all consumers
(FormulaVertex, ColumnIndex) are force-updated, then the array is
released while the logical version remains monotonically increasing.

- **`UndoRedo.oldData`** grew linearly even when entries were evicted
from the undo stack. Fixed by tracking which LTAS versions each
`UndoEntry` references (`getReferencedOldDataVersions()`), cleaning up
on eviction/clear, guarding against writes when `undoLimit === 0`, and
running orphan cleanup after compaction to handle a race condition where
lazy-apply re-inserts already-evicted keys.

### Changed files

| File | Change |
|------|--------|
| `LazilyTransformingAstService.ts` | `versionOffset`, `compact()`,
`needsCompaction()` with threshold=50, offset-aware iteration |
| `UndoRedo.ts` | `getReferencedOldDataVersions()` on interface + 7
subclasses, eviction cleanup, `undoLimit===0` guard,
`cleanupOrphanedOldData()`, `forceApply` parity in
`undoMoveRows`/`undoMoveColumns` |
| `HyperFormula.ts` | Compaction trigger in
`recomputeIfDependencyGraphNeedsIt()` |
| `ColumnIndex.ts` | `forceApplyPostponedTransformations()` — iterates
all ValueIndex entries |
| `ColumnBinarySearch.ts` | No-op `forceApplyPostponedTransformations()`
|
| `SearchStrategy.ts` | New method on `ColumnSearchStrategy` interface |
| `Operations.ts` | Added `columnSearch.forceApply` to undo path (no
compact — centralized in HyperFormula.ts) |

### What is NOT fixed here

- **Parser cache** (`ParserWithCaching`) — unbounded growth tracked
separately in #1635

## Test plan

- 18 dedicated tests in `hyperformula-tests` — see companion PR in that
repo
- Full test suite: **480 suites / 5396 tests passed**
- Benchmark validated threshold=50 as optimal (eager compaction is ~18×
slower on a 2.5k formula sheet)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Touches core recalculation/transform and undo/redo paths; while aimed
at memory safety, compaction/cleanup timing could affect formula
correctness or undo behavior in edge cases.
> 
> **Overview**
> Prevents unbounded memory growth in long-running engines by **adding
threshold-based compaction** of lazy formula transformations and by
**cleaning up undo snapshot (`oldData`) entries** when undo/redo stack
entries are cleared or evicted.
> 
> Introduces new config `maxPendingLazyTransformations` (default `50`)
and wires it into engine construction; when the threshold is reached,
`HyperFormula` forces pending transformations to be applied (dependency
graph + column search), compacts the transformation history, and prunes
orphaned `UndoRedo.oldData`. Column search strategies now expose
`forceApplyPostponedTransformations()` (real implementation for
`ColumnIndex`, no-op for binary search), and undo for move operations
ensures postponed transformations are applied before restoring old data.
Documentation and changelog are updated accordingly.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d8ebe1d. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Kuba Sekowski <jakub.sekowski@handsontable.com>

Author: marcin-kordas-hoc

CHANGELOG.md

@@ -9,10 +9,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Added
 
+- Added `maxPendingLazyTransformations` configuration option to control memory usage by limiting accumulated transformations before cleanup. [#1629](https://github.com/handsontable/hyperformula/issues/1629)
 - Added a new function: TEXTJOIN. [#1640](https://github.com/handsontable/hyperformula/pull/1640)
 
 ### Fixed
 
+- Fixed a memory leak in `LazilyTransformingAstService` where the transformations array grew unboundedly, causing increasing memory usage over time. [#1629](https://github.com/handsontable/hyperformula/issues/1629)
+- Fixed a memory leak in `UndoRedo` where `oldData` entries for evicted undo stack entries were never cleaned up, causing increasing memory usage over time. [#1629](https://github.com/handsontable/hyperformula/issues/1629)
 - Fixed the IRR function returning `#NUM!` error when the initial investment significantly exceeds the sum of returns. [#1628](https://github.com/handsontable/hyperformula/issues/1628)
 - Fixed the ADDRESS function ignoring `defaultValue` when arguments are syntactically empty (e.g., `=ADDRESS(2,3,,FALSE())`). [#1632](https://github.com/handsontable/hyperformula/issues/1632)
 

docs/guide/performance.md

@@ -37,6 +37,29 @@ cells filled, but located very far from each other.
 the fill ratio of the sheet. Let the engine choose the best strategy
 for you.
 
+## Lazy transformation cleanup
+
+Structural operations (adding/removing rows/columns, moving cells) create
+transformations that are applied lazily to formulas. Over time, these
+transformations accumulate in memory. HyperFormula automatically flushes
+them when their count reaches the `maxPendingLazyTransformations` threshold
+(default: 50).
+
+You can tune this setting to balance memory usage and CPU overhead:
+
+* **Lower values** (e.g., 10) reduce peak memory usage but trigger
+  cleanup more frequently, adding slight CPU overhead per flush.
+* **Higher values** (e.g., 200) reduce the frequency of cleanup but
+  allow more memory to accumulate between flushes.
+* The default of **50** works well for most use cases.
+
+```javascript
+const hf = HyperFormula.buildEmpty({
+  licenseKey: 'gpl-v3',
+  maxPendingLazyTransformations: 100,
+})
+```
+
 ## Suspending automatic recalculations
 
 By default, HyperFormula recalculates formulas after every change.

src/BuildEngineFactory.ts

@@ -72,7 +72,7 @@ export class BuildEngineFactory {
 
     const namedExpressions = new NamedExpressions()
     const functionRegistry = new FunctionRegistry(config)
-    const lazilyTransformingAstService = new LazilyTransformingAstService(stats)
+    const lazilyTransformingAstService = new LazilyTransformingAstService(stats, config.maxPendingLazyTransformations)
     const dependencyGraph = DependencyGraph.buildEmpty(lazilyTransformingAstService, config, functionRegistry, namedExpressions, stats)
     const columnSearch = buildColumnSearchStrategy(dependencyGraph, config, stats)
     const sheetMapping = dependencyGraph.sheetMapping

src/Config.ts

@@ -62,6 +62,7 @@ export class Config implements ConfigParams, ParserConfig {
     timeFormats: ['hh:mm', 'hh:mm:ss.sss'],
     thousandSeparator: '',
     undoLimit: 20,
+    maxPendingLazyTransformations: 50,
     useRegularExpressions: false,
     useWildcards: true,
     useColumnIndex: false,
@@ -135,6 +136,8 @@ export class Config implements ConfigParams, ParserConfig {
   /** @inheritDoc */
   public readonly undoLimit: number
   /** @inheritDoc */
+  public readonly maxPendingLazyTransformations: number
+  /** @inheritDoc */
   public readonly context: unknown
 
   /**
@@ -198,6 +201,7 @@ export class Config implements ConfigParams, ParserConfig {
       useArrayArithmetic,
       useStats,
       undoLimit,
+      maxPendingLazyTransformations,
       useColumnIndex,
       useRegularExpressions,
       useWildcards,
@@ -244,10 +248,12 @@ export class Config implements ConfigParams, ParserConfig {
     this.nullDate = configValueFromParamCheck(nullDate, instanceOfSimpleDate, 'IDate', 'nullDate')
     this.leapYear1900 = configValueFromParam(leapYear1900, 'boolean', 'leapYear1900')
     this.undoLimit = configValueFromParam(undoLimit, 'number', 'undoLimit')
+    this.maxPendingLazyTransformations = configValueFromParam(maxPendingLazyTransformations, 'number', 'maxPendingLazyTransformations')
     this.useRegularExpressions = configValueFromParam(useRegularExpressions, 'boolean', 'useRegularExpressions')
     this.useWildcards = configValueFromParam(useWildcards, 'boolean', 'useWildcards')
     this.matchWholeCell = configValueFromParam(matchWholeCell, 'boolean', 'matchWholeCell')
     validateNumberToBeAtLeast(this.undoLimit, 'undoLimit', 0)
+    validateNumberToBeAtLeast(this.maxPendingLazyTransformations, 'maxPendingLazyTransformations', 1)
     this.maxRows = configValueFromParam(maxRows, 'number', 'maxRows')
     validateNumberToBeAtLeast(this.maxRows, 'maxRows', 1)
     this.maxColumns = configValueFromParam(maxColumns, 'number', 'maxColumns')

src/ConfigParams.ts

@@ -402,6 +402,21 @@ export interface ConfigParams {
    * @category Undo and Redo
    */
   undoLimit: number,
+  /**
+   * Controls memory usage for long-running instances by limiting the number of
+   * pending lazy transformations before cleanup occurs.
+   *
+   * Structural operations (adding/removing rows/columns, moving cells) create
+   * transformations that are applied lazily to formulas. This setting determines
+   * how many can accumulate before they are flushed and memory is reclaimed.
+   *
+   * Lower values reduce peak memory usage but may slightly increase CPU overhead.
+   * Higher values reduce overhead but allow more memory accumulation.
+   *
+   * @default 50
+   * @category Engine
+   */
+  maxPendingLazyTransformations: number,
   /**
    * When set to `true`, criteria in functions (SUMIF, COUNTIF, ...) are allowed to use regular expressions.
    * @default false

src/HyperFormula.ts

@@ -4595,6 +4595,20 @@ export class HyperFormula implements TypedEmitter {
     this._functionRegistry = newEngine.functionRegistry
   }
 
+  /**
+   * When enough transformations have accumulated, forces all formula vertices and
+   * column index entries to apply pending lazy transformations, then compacts the
+   * transformation history and cleans up orphaned undo oldData entries.
+   */
+  private compactLazyTransformationsIfNeeded(): void {
+    if (this._lazilyTransformingAstService.needsCompaction()) {
+      this._dependencyGraph.forceApplyPostponedTransformations()
+      this._columnSearch.forceApplyPostponedTransformations()
+      this._lazilyTransformingAstService.compact()
+      this._lazilyTransformingAstService.undoRedo?.cleanupOrphanedOldData()
+    }
+  }
+
   /**
    * Runs a recomputation starting from recently changed vertices.
    *
@@ -4610,6 +4624,8 @@ export class HyperFormula implements TypedEmitter {
       const verticesToRecomputeFrom = this.dependencyGraph.verticesToRecompute()
       this.dependencyGraph.clearDirtyVertices()
 
+      this.compactLazyTransformationsIfNeeded()
+
       if (verticesToRecomputeFrom.length > 0) {
         changes.addAll(this.evaluator.partialRun(verticesToRecomputeFrom))
       }

src/LazilyTransformingAstService.ts

@@ -11,21 +11,56 @@ import {StatType} from './statistics'
 import {Statistics} from './statistics/Statistics'
 import {UndoRedo} from './UndoRedo'
 
+/**
+ * Manages lazy application of formula AST transformations.
+ *
+ * ## Problem
+ * Structural operations (adding/removing rows/columns, moving cells, renaming sheets)
+ * require updating every formula that references the affected area. Applying these
+ * transformations eagerly to all formulas after every operation is expensive, especially
+ * for large spreadsheets with many formulas.
+ *
+ * ## Solution: Lazy Transformation
+ * Instead of transforming all formulas immediately, this service stores transformations
+ * in a queue. Each formula vertex (FormulaVertex) and column index entry (ValueIndex)
+ * tracks its own version number. When a consumer needs up-to-date data, it calls
+ * `applyTransformations()` with its current version and receives all transformations
+ * accumulated since that version.
+ *
+ * ## Compaction
+ * Over time, the transformations array grows unboundedly. To prevent this memory leak,
+ * the engine periodically triggers compaction when the number of accumulated
+ * transformations reaches the configurable `maxPendingLazyTransformations`:
+ *
+ * 1. All FormulaVertex instances are forced to apply pending transformations
+ *    (via `DependencyGraph.forceApplyPostponedTransformations()`).
+ * 2. All ColumnIndex entries are forced to apply pending transformations
+ *    (via `ColumnSearchStrategy.forceApplyPostponedTransformations()`).
+ * 3. `compact()` is called, which advances `versionOffset` and clears the
+ *    transformations array.
+ * 4. `UndoRedo.cleanupOrphanedOldData()` removes any oldData entries that were
+ *    written during forced application but belong to already-evicted undo entries.
+ *
+ * The `versionOffset` ensures that version numbers remain globally consistent
+ * after compaction: `version() = versionOffset + transformations.length`.
+ */
 export class LazilyTransformingAstService {
 
   public parser?: ParserWithCaching
   public undoRedo?: UndoRedo
 
   private transformations: FormulaTransformer[] = []
+  private versionOffset: number = 0
   private combinedTransformer?: CombinedTransformer
 
   constructor(
     private readonly stats: Statistics,
+    private readonly maxPendingLazyTransformations: number,
   ) {
   }
 
   public version(): number {
-    return this.transformations.length
+    return this.versionOffset + this.transformations.length
   }
 
   public addTransformation(transformation: FormulaTransformer): number {
@@ -53,8 +88,9 @@ export class LazilyTransformingAstService {
   public applyTransformations(ast: Ast, address: SimpleCellAddress, version: number): [Ast, SimpleCellAddress, number] {
     this.stats.start(StatType.TRANSFORM_ASTS_POSTPONED)
 
-    for (let v = version; v < this.transformations.length; v++) {
-      const transformation = this.transformations[v]
+    const currentVersion = this.version()
+    for (let v = Math.max(version, this.versionOffset); v < currentVersion; v++) {
+      const transformation = this.transformations[v - this.versionOffset]
       if (transformation.isIrreversible()) {
         this.undoRedo!.storeDataForVersion(v, address, this.parser!.computeHashFromAst(ast))
         this.parser!.rememberNewAst(ast)
@@ -67,15 +103,37 @@ export class LazilyTransformingAstService {
     const cachedAst = this.parser!.rememberNewAst(ast)
 
     this.stats.end(StatType.TRANSFORM_ASTS_POSTPONED)
-    return [cachedAst, address, this.transformations.length]
+    return [cachedAst, address, currentVersion]
   }
 
   public* getTransformationsFrom(version: number, filter?: (transformation: FormulaTransformer) => boolean): IterableIterator<FormulaTransformer> {
-    for (let v = version; v < this.transformations.length; v++) {
-      const transformation = this.transformations[v]
+    const currentVersion = this.version()
+    for (let v = Math.max(version, this.versionOffset); v < currentVersion; v++) {
+      const transformation = this.transformations[v - this.versionOffset]
       if (!filter || filter(transformation)) {
         yield transformation
       }
     }
   }
+
+  /**
+   * Returns true when enough transformations have accumulated to justify the cost
+   * of forcing all consumers (FormulaVertex, ColumnIndex) to apply pending changes.
+   */
+  public needsCompaction(): boolean {
+    return this.transformations.length >= this.maxPendingLazyTransformations
+  }
+
+  /**
+   * Compacts the transformations array by discarding all entries that have already
+   * been applied by every consumer. Safe to call only after all FormulaVertex and
+   * ColumnIndex consumers have been brought up to the current version.
+   * After calling, UndoRedo.cleanupOrphanedOldData() must be invoked to remove
+   * oldData entries written during forceApplyPostponedTransformations for
+   * already-evicted undo entries.
+   */
+  public compact(): void {
+    this.versionOffset += this.transformations.length
+    this.transformations = []
+  }
 }

src/Lookup/ColumnBinarySearch.ts

@@ -53,6 +53,16 @@ export class ColumnBinarySearch extends AdvancedFind implements ColumnSearchStra
   public removeValues(range: IterableIterator<[RawScalarValue, SimpleCellAddress]>): void {
   }
 
+  /**
+   * No-op: ColumnBinarySearch reads cell values directly from the dependency graph
+   * on every lookup, so it has no cached data that could become stale.
+   * Unlike ColumnIndex, which maintains a separate value-to-address index that
+   * must be kept in sync with lazy transformations, binary search always operates
+   * on the current graph state.
+   */
+  public forceApplyPostponedTransformations(): void {
+  }
+
   /*
    * WARNING: Finding lower/upper bounds in unordered ranges is not supported. When ordering === 'none', assumes matchExactly === true
    */

src/Lookup/ColumnIndex.ts

@@ -184,6 +184,24 @@ export class ColumnIndex implements ColumnSearchStrategy {
     this.index.delete(sheetId)
   }
 
+  /**
+   * Forces all ValueIndex entries to apply any pending lazy transformations,
+   * bringing every entry up to the current LazilyTransformingAstService version.
+   * Must be called before compacting LazilyTransformingAstService.
+   */
+  public forceApplyPostponedTransformations(): void {
+    for (const [sheet, sheetIndex] of this.index) {
+      sheetIndex.forEach((columnMap, col) => {
+        if (!columnMap) {
+          return
+        }
+        for (const value of columnMap.keys()) {
+          this.ensureRecentData(sheet, col, value)
+        }
+      })
+    }
+  }
+
   public getColumnMap(sheet: number, col: number): ColumnMap {
     if (!this.index.has(sheet)) {
       this.index.set(sheet, [])

src/Lookup/SearchStrategy.ts

@@ -51,6 +51,13 @@ export interface ColumnSearchStrategy extends SearchStrategy {
   moveValues(range: IterableIterator<[RawScalarValue, SimpleCellAddress]>, toRight: number, toBottom: number, toSheet: number): void,
 
   removeValues(range: IterableIterator<[RawScalarValue, SimpleCellAddress]>): void,
+
+  /**
+   * Forces all lazily-tracked ValueIndex entries to apply any pending transformations,
+   * bringing every entry's version up to the current LazilyTransformingAstService version.
+   * Must be called before compacting LazilyTransformingAstService.
+   */
+  forceApplyPostponedTransformations(): void,
 }
 
 export function buildColumnSearchStrategy(dependencyGraph: DependencyGraph, config: Config, statistics: Statistics): ColumnSearchStrategy {