Skip to content

Fix user-scoped memory consolidation#1112

Open
wayyoungboy wants to merge 1 commit into
oceanbase:mainfrom
wayyoungboy:codex/issue-1109-fix-dashboard-user-filter
Open

Fix user-scoped memory consolidation#1112
wayyoungboy wants to merge 1 commit into
oceanbase:mainfrom
wayyoungboy:codex/issue-1109-fix-dashboard-user-filter

Conversation

@wayyoungboy

Copy link
Copy Markdown
Member

Summary

  • add adapter-side scope and logical-filter validation for memory search/list results
  • avoid reporting intelligent UPDATE/DELETE actions when storage rejects them by scope
  • cover user-scope leakage, logical filters, simple-backend pushdown, and sync/async rejected updates

Validation

  • python -m pytest tests/unit/test_issue_1109_user_scope.py tests/unit/test_list_memory_filters.py tests/unit/test_oceanbase_native_filter.py -q
  • python -m pytest tests/integration/test_noop_embedding_mode.py tests/integration/test_storage_integration.py tests/integration/test_async_memory_integration.py -q
  • Local SQLite E2E: forced search to return another user's candidate during intelligent add; verified the existing user memory was unchanged and the current user received an ADD result.

Closes #1109.

@wayyoungboy
wayyoungboy marked this pull request as ready for review June 29, 2026 15:06

@lightzt99 lightzt99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix. CI is green and the sync/async + SQLite/OceanBase + nested AND/OR test coverage is solid.

Before merging, please consider the following:

P1 — Two fail-open paths conflict with this PR's goal (scope isolation)

  1. $nin silently passes all rows when op_value is not a list/tuple/set. _filter_value_matches only checks isinstance(op_value, (list, tuple, set)) and actual in op_value → return False; when isinstance fails it falls through to the trailing return True. $in is fail-closed (not isinstance(...) or actual not in op_value → return False); $nin should be symmetric.
  2. _memory_matches_filter_expression: if not isinstance(filters, dict): return True. A stray string/number passed as filters would let every memory through. Should return False (fail-closed) or raise.

P1 — search_memories overscan cap returns fewer than limit for selective filters

search_limit = max(limit * 5, 100). With 10k rows, 1% selectivity (100 matches), limit=10: overscan 100, Python filters to 1, return 1 — but 100 actually match and the user asked for 10. get_all_memories uses limit=None (full scan) so it doesn't have this bug, but search_memories shouldn't under-return.

P2 — Unknown filter operators silently drop all rows

_filter_value_matches returns False for $regex / $exists / $between / $all. Users get an empty result with no warning, believing the filter worked. Suggest raise ValueError(f"Unsupported filter operator: {op}").

P2 — like / $like anchor semantics lost

str(op_value).replace("%", "") substring match collapses like "private%" (prefix), like "%private" (suffix), and like "%private%" (contains) to the same contains behavior. SQL LIKE anchors silently don't work — risky for permission/scope filters. Either implement SQL semantics or document "substring contains" only.

P2 — DELETE rejection path untested

Tests cover UPDATE rejection (sync + async), but no test verifies that _intelligent_add correctly omits the result when delete is rejected by scope. Add a sync + async DELETE-rejected case.

P3 — count_all_memories full-table scan has no upper bound

filters_fully_pushed=Falselen(get_all_memories(limit=None)) pulls every matching row into Python. Dashboard calls count to show totals — at 100k memories a single count is O(N). Suggest a cap (e.g. 100k) or document the limitation.

P3 — Warning message is inaccurate

_intelligent_add logs "Skipped UPDATE for memory %s because it is outside the requested scope", but _update_memory returning None may also mean the memory was concurrently deleted or never existed. Suggest "not found or outside requested scope".

P3 — Single 740-line commit is hard to review/rollback

Three independent changes (_intelligent_add truthy return / search-list defensive filter / OceanBase logical filter translation) are bundled. Splitting into 3 commits would help.

Band-aid nature

The defensive re-check at the search/list tail is a patch, not a root-cause fix. If the real cause is SQLite list() not pushing user_id filters, the next new store path will bypass this defense too. Worth a follow-up to fix the backend's missing filter; keep this layer as defense-in-depth.

Summary: direction is right; please fix the two P1 fail-open paths and the overscan cap before merge, the rest can be follow-ups.

@wayyoungboy
wayyoungboy force-pushed the codex/issue-1109-fix-dashboard-user-filter branch from cc38788 to 32f6b95 Compare July 4, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: mixed usage of user id in the powermem dashboard

2 participants