Skip to content

TiMem tree consolidation: singleton leaves can permanently starve newer leaves from being swept #6393

Description

@bug-ops

Description

load_tree_leaves_unconsolidated (crates/zeph-memory/src/store/memory_tree.rs) loads the oldest batch_size leaves with parent_id IS NULL for each consolidation sweep. Leaves that don't cluster with anything (singletons) never get a parent_id assigned, so once more than batch_size singleton leaves accumulate, every subsequent sweep keeps re-loading and re-embedding the same stuck set of oldest leaves — newer leaves with real memorable content never get a chance to be considered for clustering.

This bug pre-dates this issue and was unreachable in practice because insert_tree_leaf had zero production callers (see #6384) — the tree was always empty, so the starvation condition could never trigger. #6384's fix wires a real producer into persist_message, which makes this reachable in any long-running deployment once more than batch_size singleton leaves accumulate.

Flagged during PR #6392's implementation critique (impl-critic) as out of scope for that bug fix.

Expected Behavior

Leaves that fail to cluster after some number of sweep attempts (or age threshold) should not permanently block newer leaves from being considered — e.g. via a bounded retry count, an age-based skip/requeue, or paginating past already-attempted singletons within a sweep.

Actual Behavior

load_tree_leaves_unconsolidated's unconditional oldest-first batch_size load has no starvation guard, so a batch of never-clustering singleton leaves can permanently monopolize every sweep.

Environment

Suggested Fix Direction

Track a per-leaf attempt count or last-attempted timestamp, and either skip leaves that have exceeded a retry threshold in the load query, or bias the load toward less-recently-attempted leaves rather than strict oldest-first.

Metadata

Metadata

Assignees

Labels

P3Research — medium-high complexitybugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions