You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CDP treats member identities as case-insensitive when resolving people (`lower(value)` in lookups and many DAL queries), which matches how major platforms behave:
11
+
12
+
-**GitHub / GitLab**: usernames are case-insensitive for uniqueness, but case-preserving for display (`WillsonHG` and `willsonhg` are the same account; APIs return preferred casing).
13
+
-**Discord** (new usernames): forced lowercase.
14
+
-**Email**: stored and compared lowercase in practice.
15
+
16
+
Historically, `memberIdentities` uniqueness was defined on raw `value` (case-sensitive):
Lookups used `lower(value)`, but inserts often did not. Data-sink `mergeData` matched with exact `value ===`, so a self-serve lowercase `willsonhg` plus a later GitHub ingest of `WillsonHG` produced two rows for the same identity — often both verified on the same member. Prod had tens of thousands of GitHub case-variant groups.
22
+
23
+
Ticket CM-1349 proposed soft-deleting / auto-verifying case variants at verification time. That treats a write-path bug as a product special case.
24
+
25
+
## Decision
26
+
27
+
**Mental model**
28
+
29
+
| Kind | Store | Compare / unique on |
30
+
| --- | --- | --- |
31
+
| username | preferred casing from the source/integration |`lower(value)`|
32
+
| email | always lowercase |`lower(value)` (same as stored) |
33
+
34
+
Identity equality in CDP is `(platform, type, lower(value))`. The `value` column keeps what the source sent for usernames; we do not rewrite GitHub `login` casing on ingest.
35
+
36
+
**Enforcement**
37
+
38
+
1.**Write paths** match and upsert with case-insensitive equality (`isSameMemberIdentity` / `lower(value)`). Do not insert a second row that only differs by casing.
39
+
2.**DB uniqueness** uses expression unique indexes on `lower(value)` (partial on `deletedAt is null`, and verified-only for the global verified owner index). See migration `V1785255019__member_identities_case_insensitive_unique_indexes.sql`.
40
+
3.**Existing duplicates** are cleaned with a one-time script before the unique indexes can be applied: same-member case variants → keep one (prefer verified + `verifiedBy`, else most recent integration casing) and soft-delete the rest; cross-member unverified variants of a verified identity → soft-delete the unverified; both verified across members → merge / existing capitalization-merge workflows, not blind soft-delete.
41
+
4.**Verification** does not need special “soft-delete case siblings” logic once the invariant holds — verifying finds the one row.
42
+
43
+
## Alternatives Considered
44
+
45
+
### Alternative 1: Soft-delete / auto-verify case variants at identity verification time (CM-1349 as written)
46
+
47
+
-**Pros**: Fixes the user-visible self-serve pain quickly; no schema change.
48
+
-**Cons**: Case variants keep being inserted by ingest/enrichment; verify path becomes a mop; duplicates still break uniqueness and analytics.
49
+
-**Why not**: Papers over the root cause. If case variants should not exist, stop creating them and clean existing data.
50
+
51
+
### Alternative 2: Always store usernames lowercase (like emails / Discord)
52
+
53
+
-**Pros**: Simplest storage; uniqueness on `value` works without expression indexes.
54
+
-**Cons**: Throws away GitHub/GitLab preferred casing; diverges from source payloads; confuses display and support (“CDP shows lowercase but GitHub shows mixed”).
55
+
-**Why not**: We want GitHub-style case-preserving storage. Uniqueness belongs on `lower(value)`, not on mutating the stored handle.
56
+
57
+
### Alternative 3: Keep case-sensitive unique indexes; only fix app-layer matching
58
+
59
+
-**Pros**: No migration; no cleanup required to change indexes.
60
+
-**Cons**: App bugs or races can still insert case variants; DB does not enforce the domain invariant.
61
+
-**Why not**: At this scale, durable invariants need to live in the database, not only in callers.
62
+
63
+
### Alternative 4: Update stored casing on every ingest when preferred casing differs
-**Cons**: Unsafe while same-member case-variant pairs still exist (updating both rows to the same `value` hits the old unique index). Extra write noise.
67
+
-**Why not**: Deferred until after cleanup. Preventing duplicate inserts is enough for the durable fix; optional casing refresh can come later.
68
+
69
+
## Consequences
70
+
71
+
### Positive
72
+
73
+
- One clear rule: same platform + type + lower(value) ⇒ same identity.
- Preferred username casing from integrations is preserved.
77
+
78
+
### Negative
79
+
80
+
- Cleanup must run before the unique-index migration, or `create unique index` fails (and can leave an `INVALID` index).
81
+
- Expression unique indexes are slightly less obvious than column-only uniques; callers must keep using `lower(value)` (or `isSameMemberIdentity`) consistently.
82
+
- Conflict handlers need to recognize both old and new constraint names during rollout.
83
+
84
+
### Risks
85
+
86
+
-**Migration applied before cleanup** — mitigated by documenting order: write-path fix → cleanup script → unique-index migration.
87
+
-**Cross-member verified case variants** — rare; require merge, not soft-delete. Existing `findAndMergeMembersWithSamePlatformIdentitiesDifferentCapitalization` covers part of this.
88
+
-**Incomplete write-path coverage** — mitigated by DB unique indexes as the backstop once cleanup is done; shared `isSameMemberIdentity` for app equality.
0 commit comments