fix: memberToMerge and memberToMergeRaw fall out of sync (CM-1339)

[joanagmaia]

Summary

removeMemberToMerge in data-access-layer/src/member_merge/index.ts only deleted from memberToMerge. After a merge the corresponding memberToMergeRaw row was never removed, leaving the pair eligible to be re-queued.

addMemberToMerge in the enrichment worker DAL was also investigated as a potential source of orphans, but turned out to be dead code with no callers — no fix needed there.

DB findings at time of discovery

Metric	Count
memberToMergeRaw total rows	47,647,783
memberToMergeRaw rows with similarity > 0.75	164,424
memberToMerge total rows	124,892
Orphans in memberToMerge with no raw entry	5,253
Raw rows above threshold missing from memberToMerge	46,093

Changes

• services/libs/data-access-layer/src/member_merge/index.ts — removeMemberToMerge now also deletes from memberToMergeRaw

Jira: CM-1339

[cursor]

PR Summary

Medium Risk
Touches member-merge persistence used on manual merge API paths; wrong pairing could delete unrelated raw rows, though the change mirrors the existing memberToMerge delete predicate.

Overview
removeMemberToMerge now removes matching rows from memberToMergeRaw as well as memberToMerge, so manual merge / API paths that clear a suggestion no longer leave stale raw-table rows behind.

This aligns removal with flows that already update both tables (e.g. LLM merge processing) and addresses memberToMerge / memberToMergeRaw drift described in CM-1339.

^{Reviewed by Cursor Bugbot for commit 97fd8b9. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

This PR fixes consistency issues between the filtered merge-suggestion table (memberToMerge) and its raw source table (memberToMergeRaw) so that merge candidates don’t get re-queued indefinitely and inserts don’t leave orphaned rows.

Changes:

• Extend removeMemberToMerge to also delete the corresponding entry from memberToMergeRaw.
• Extend the enrichment worker’s addMemberToMerge to also insert into memberToMergeRaw, and add ON CONFLICT DO NOTHING to both inserts (also removing a stray ").

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
services/libs/data-access-layer/src/old/apps/members_enrichment_worker/index.ts	Inserts merge candidates into both memberToMerge and memberToMergeRaw (with conflict handling).
services/libs/data-access-layer/src/member_merge/index.ts	Cleans up memberToMergeRaw when removing a merge candidate from memberToMerge.

Comments suppressed due to low confidence (2)

services/libs/data-access-layer/src/old/apps/members_enrichment_worker/index.ts:356

• Both INSERTs omit "createdAt"/"updatedAt". Per the Flyway schema for "memberToMerge" these columns are NOT NULL (see backend/src/database/migrations/V1666966941__initial.sql:305-310). If the DB doesn’t have defaults/triggers, this statement will fail at runtime. Even if it currently works, explicitly setting timestamps here keeps the insert portable and consistent with other codepaths (e.g. backend memberRepository inserts NOW(), NOW()).

         AND mi.value in ($(values:csv))
         AND mi.type = $(type)
         AND EXISTS (SELECT 1 FROM "memberSegments" ms WHERE ms."memberId" = mi."memberId")`,

services/libs/data-access-layer/src/old/apps/members_enrichment_worker/index.ts:362

• This INSERT into "memberToMergeRaw" also omits NOT NULL "createdAt"/"updatedAt" columns (see backend/src/database/migrations/V1715071755__rawSuggestionTables.sql:1-9). Without defaults/triggers, it will fail and prevent enqueueing merge suggestions. Consider inserting timestamps explicitly (matching how other merge-suggestion inserts populate NOW(), NOW()).

      platform,
      values,
      type,
    },

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.