Profile trait history: fixes + surface traits/history in UI#539
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📄 Knowledge review✏️ Documentation updates1 page was updated by changes in this PR.
📝 Profile Trait Change Tracking — changes@@ -18,8 +18,8 @@
| Column | Type | Description |
|---|---|---|
| `id` | `text` (UUID) | Primary key, generated with `crypto.randomUUID()` |
-| `website_id` | `text` | Foreign key → `websites.id` (cascade delete) |
-| `profile_id` | `text` | The profile this change belongs to |
+| `website_id` | `text` | Composite foreign key → `profiles.(website_id, profile_id)` (cascade delete) |
+| `profile_id` | `text` | Composite foreign key → `profiles.(website_id, profile_id)` (cascade delete) |
| `traits` | `jsonb` | Full merged non-PII trait snapshot **after** this change |
| `changes` | `jsonb` | Diff object: `{ [traitKey]: { old, new } }` for every key that changed |
| `source` | `text` | Origin of the change — `"identify"` or `"billing"` |
@@ -41,7 +41,7 @@
Although trait history is append-only time-series data, it is stored in Postgres for three reasons:
1. **Transactional consistency** — the change record is written inside the same database transaction as the profile upsert, so the profile and its history are always in sync.
-2. **Trivial GDPR deletes** — deleting a website cascades to all of its trait history automatically; no cross-database coordination is needed.
+2. **Trivial GDPR deletes** — deleting a profile cascades to all of its trait history automatically; no cross-database coordination is needed.
3. **Events already carry traits** — Databuddy events store the traits that were active at event time, so there is no need to join against a separate time-series store to reconstruct the state at any past moment.
### Indexes
@@ -92,11 +92,12 @@
Trait history is recorded inside `upsertProfile` in [`packages/services/src/identity.ts`](https://github.com/databuddy-analytics/Databuddy/blob/main/packages/services/src/identity.ts). When the call includes trait updates, the following steps execute inside a single Postgres transaction:
-1. **`SELECT … FOR UPDATE`** — the existing profile row is read with a row-level lock to prevent concurrent upserts from interleaving their history records.
-2. **`INSERT … ON CONFLICT DO UPDATE`** — the profile is created or its traits are merged in place.
-3. **`applyTraits`** — the diff is computed against the traits fetched in step 1.
-4. **Conditional insert** — if `changes.length > 0`, a new row is inserted into `profile_trait_changes` containing the diff, the merged trait snapshot, and the source.
-5. **Return** — the function returns the `ProfileTraitUpdate` (or `null` if there were no trait-level changes to process).
+1. **`pg_advisory_xact_lock`** — an advisory lock is acquired on the `(websiteId, profileId)` pair to serialize concurrent first-time identifies. `FOR UPDATE` can't lock a not-yet-existent row, so this prevents race conditions when two concurrent identify calls try to create history for the same profile that doesn't exist yet.
+2. **Read existing traits** — the profile's current trait snapshot is read (without a row lock, since the advisory lock provides serialization).
+3. **`INSERT … ON CONFLICT DO UPDATE`** — the profile is created or its traits are merged in place.
+4. **`applyTraits`** — the diff is computed against the traits fetched in step 2.
+5. **Conditional insert** — if `changes.length > 0`, a new row is inserted into `profile_trait_changes` containing the diff, the merged trait snapshot, and the source.
+6. **Return** — the function returns the `ProfileTraitUpdate` (or `null` if there were no trait-level changes to process).
Because the history record is written in the same transaction, it is impossible for the profile to be updated without a corresponding history entry, or for a history entry to exist whose state does not match what the profile row holds.
@@ -171,21 +172,58 @@
## Querying Profile History
-### The `get_profile_history` AI Tool
-
-Profile trait history is queryable through the `get_profile_history` tool, registered in [`packages/ai/src/ai/tools/profiles.ts`](https://github.com/databuddy-analytics/Databuddy/blob/main/packages/ai/src/ai/tools/profiles.ts) inside `buildProfileTools`. The tool is available to both the **in-app agent** and the **MCP** integration, and website-access enforcement is applied automatically via `opts.resolveSite`.
+### The `profiles.getHistory` RPC Endpoint
+
+Profile trait history is exposed through the `profiles.getHistory` RPC endpoint, defined in [`packages/rpc/src/routers/profiles.ts`](https://github.com/databuddy-analytics/Databuddy/blob/main/packages/rpc/src/routers/profiles.ts). The endpoint is available to authenticated users with website read permission and returns the trait change timeline for a specific profile, newest first.
+
+#### Route
+
+```
+POST /profiles/getHistory
+```
#### Parameters
| Parameter | Type | Description |
|---|---|---|
| `websiteId` | `string` | The website to query (access-checked) |
-| `profileId` | `string` | The profile whose trait history to retrieve |
+| `profileId` | `string` | The profile whose trait history to retrieve (min: 1, max: 128 characters) |
| `limit` | `number` | Number of entries to return. Min: `1`, max: `200`, default: `50` |
#### Response
-The tool returns an object with two fields:
+Returns an array of trait change entries, ordered **most recent first**. Each entry provides:
+
+```ts
+Array<{
+ changes: Record<string, { old: TraitValue; new: TraitValue }>;
+ traits: Record<string, unknown>;
+ source: string;
+ createdAt: Date;
+}>
+```
+
+- `changes` — the diff showing exactly which traits changed and their old/new values.
+- `traits` — the complete trait snapshot immediately after this change.
+- `source` — `"identify"` or `"billing"`.
+- `createdAt` — the timestamp of the change with millisecond precision.
+
+Because each row carries the full snapshot, answering "what were the traits at time T" requires only selecting the first entry with `createdAt <= T` — no diff accumulation is needed.
+
+#### Example Usage
+
+The endpoint is designed to answer questions like:
+
+- **"When did the plan change?"** — filter the response array for entries where `changes.plan` exists.
+- **"What were the traits at time T?"** — take the first entry with `createdAt <= T`.
+- **"Show me all changes to the email trait"** — filter for entries where `changes.email` is present.
+- **"How many times has this profile been updated?"** — use the length of the returned array.
+
+### The `get_profile_history` AI Tool
+
+Profile trait history is also queryable through the `get_profile_history` tool, registered in [`packages/ai/src/ai/tools/profiles.ts`](https://github.com/databuddy-analytics/Databuddy/blob/main/packages/ai/src/ai/tools/profiles.ts) inside `buildProfileTools`. The tool is available to both the **in-app agent** and the **MCP** integration, and website-access enforcement is applied automatically via `opts.resolveSite`.
+
+The tool accepts the same parameters as the RPC endpoint and returns an object with two fields:
```ts
{
@@ -199,23 +237,6 @@
}
```
-Results are ordered **most recent first**. Each entry provides:
-- `changes` — the diff showing exactly which traits changed and their old/new values.
-- `traits` — the complete trait snapshot immediately after this change.
-- `source` — `"identify"` or `"billing"`.
-- `changedAt` — the timestamp of the change with millisecond precision.
-
-Because each row carries the full snapshot, answering "what were the traits at time T" requires only selecting the first entry with `changedAt <= T` — no diff accumulation is needed.
-
-#### Example Questions
-
-The tool is designed to answer questions like:
-
-- **"When did the plan change?"** — filter `history` for entries where `changes.plan` exists.
-- **"What were the traits at time T?"** — take the first entry with `changedAt <= T`.
-- **"Show me all changes to the email trait"** — filter for entries where `changes.email` is present.
-- **"How many times has this profile been updated?"** — use `count` from the response.
-
## Use Cases
@@ -262,7 +283,7 @@
### GDPR Deletion
-Deleting a website row cascades to all `profile_trait_changes` rows for that website, because the `website_id` foreign key is defined with `ON DELETE CASCADE`. This ensures that when a customer exercises their right to erasure and their website is deleted, all associated trait history is removed in the same database operation with no additional cleanup steps required.
+Deleting a profile row cascades to all `profile_trait_changes` rows for that profile, because the `(website_id, profile_id)` composite foreign key is defined with `ON DELETE CASCADE`. This ensures that when a customer exercises their right to erasure and their profile is deleted, all associated trait history is removed in the same database operation with no additional cleanup steps required.
## Technical Reference
@@ -276,9 +297,7 @@
"profile_trait_changes",
{
id: text().primaryKey(),
- websiteId: text("website_id")
- .notNull()
- .references(() => websites.id, { onDelete: "cascade" }),
+ websiteId: text("website_id").notNull(),
profileId: text("profile_id").notNull(),
// Full merged non-PII traits after this change — state at time T is the
// latest row with createdAt <= T, no diff folding needed.
@@ -304,6 +323,11 @@
table.createdAt
),
index("profile_trait_changes_changes_gin_idx").using("gin", table.changes),
+ foreignKey({
+ columns: [table.websiteId, table.profileId],
+ foreignColumns: [profiles.websiteId, profiles.profileId],
+ name: "profile_trait_changes_profile_fkey",
+ }).onDelete("cascade"),
]
);
```
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 2 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Architecture diagram
sequenceDiagram
participant UI as Client App
participant API as Identity API
participant DB as PostgreSQL
participant Lock as Advisory Lock
Note over UI,DB: Profile Delete Flow with Trait History
UI->>API: DELETE /profiles/{websiteId}/{profileId}
API->>DB: DELETE FROM profiles WHERE website_id=w AND profile_id=p
DB->>DB: CASCADE: DELETE profile_trait_changes WHERE (website_id,profile_id)=(w,p)
DB-->>API: 200 OK
API-->>UI: Deletion confirmed
Note over UI,DB: Website Delete Flow (transitive cascade)
UI->>API: DELETE /websites/{websiteId}
API->>DB: DELETE FROM websites WHERE id=w
DB->>DB: CASCADE: DELETE profiles WHERE website_id=w
DB->>DB: CASCADE: DELETE profile_trait_changes WHERE (website_id,profile_id) IN (deleted profiles)
DB-->>API: 200 OK
API-->>UI: Deletion confirmed
Note over UI,DB: First-Time Identify Flow (concurrent safe)
UI->>API: POST /identify (websiteId, profileId, traits)
API->>DB: BEGIN TRANSACTION
API->>Lock: SELECT pg_advisory_xact_lock(hash('profile:w:p'))
alt First time (no existing profile)
DB->>API: Lock acquired
API->>DB: SELECT traits FROM profiles WHERE website_id=w AND profile_id=p LIMIT 1
DB-->>API: Empty result
API->>DB: INSERT INTO profiles (website_id, profile_id, traits)
API->>DB: INSERT INTO profile_trait_changes (website_id, profile_id, changes)
API->>DB: COMMIT
DB-->>API: Profile created with initial traits
else Existing profile
DB->>API: Lock acquired
API->>DB: SELECT traits FROM profiles WHERE website_id=w AND profile_id=p LIMIT 1
DB-->>API: Existing traits
API->>DB: UPDATE profiles SET traits = merge(traits, new_traits) WHERE website_id=w AND profile_id=p
API->>DB: INSERT INTO profile_trait_changes (website_id, profile_id, changes)
API->>DB: COMMIT
DB-->>API: Profile updated
end
API-->>UI: Identify result
Note over UI,DB: Concurrent First-Time Identify (race prevented)
UI->>API: POST /identify (w, p, traits_A)
User2->>API: POST /identify (w, p, traits_B)
API->>Lock: SELECT pg_advisory_xact_lock(hash('profile:w:p'))
User2->>Lock: SELECT pg_advisory_xact_lock(hash('profile:w:p'))
Lock-->>API: Lock acquired
API->>DB: SELECT traits... (returns empty)
API->>DB: INSERT profile + trait_history
API->>DB: COMMIT
API->>Lock: Lock released
Lock-->>User2: Lock acquired (now sees existing profile)
User2->>DB: SELECT traits... (returns traits_A)
User2->>DB: UPDATE profile + INSERT trait_history
User2->>DB: COMMIT
User2->>Lock: Lock released
API-->>UI: Identify result (traits_A)
User2-->>User2: Identify result (merged traits)
Shadow auto-approve: would require human review. Database schema change (composite FK) and transaction concurrency modification; requires human review due to potential data integrity and performance implications.
Re-trigger cubic
Contributor
Greptile SummaryThis PR addresses two data-correctness issues from the previous trait-history shipping: it replaces the standalone
Confidence Score: 4/5Safe to merge; both targeted fixes are correct and the advisory-lock replacement is sound. The two fixes are well-reasoned and correctly implemented. The one gap worth noting before closing the GDPR story is that profile_aliases still has no cascade from profiles, so a profile deletion leaves alias rows pointing at the removed profile — the same class of problem the schema fix addresses for trait history. packages/db/src/drizzle/schema/identity.ts — the profileAliases table should be checked for a matching cascade FK before GDPR deletes are considered fully covered. Important Files Changed
Sequence Diagram%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant C1 as Concurrent Identify #1
participant C2 as Concurrent Identify #2
participant PG as PostgreSQL
C1->>PG: BEGIN transaction
C1->>PG: pg_advisory_xact_lock('profile:ws:uid')
Note over PG: Lock acquired by C1
C2->>PG: BEGIN transaction
C2->>PG: pg_advisory_xact_lock('profile:ws:uid')
Note over PG: C2 blocks — same hash key
C1->>PG: "SELECT traits FROM profiles (reads {})"
C1->>PG: INSERT profiles ON CONFLICT DO UPDATE
C1->>PG: "INSERT profile_trait_changes (delta: plan=pro)"
C1->>PG: COMMIT → lock released
Note over C2: Unblocked
C2->>PG: SELECT traits FROM profiles (reads updated traits)
C2->>PG: INSERT profiles ON CONFLICT DO UPDATE (merge)
C2->>PG: INSERT profile_trait_changes (delta only if changed)
C2->>PG: COMMIT
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant C1 as Concurrent Identify #1
participant C2 as Concurrent Identify #2
participant PG as PostgreSQL
C1->>PG: BEGIN transaction
C1->>PG: pg_advisory_xact_lock('profile:ws:uid')
Note over PG: Lock acquired by C1
C2->>PG: BEGIN transaction
C2->>PG: pg_advisory_xact_lock('profile:ws:uid')
Note over PG: C2 blocks — same hash key
C1->>PG: "SELECT traits FROM profiles (reads {})"
C1->>PG: INSERT profiles ON CONFLICT DO UPDATE
C1->>PG: "INSERT profile_trait_changes (delta: plan=pro)"
C1->>PG: COMMIT → lock released
Note over C2: Unblocked
C2->>PG: SELECT traits FROM profiles (reads updated traits)
C2->>PG: INSERT profiles ON CONFLICT DO UPDATE (merge)
C2->>PG: INSERT profile_trait_changes (delta only if changed)
C2->>PG: COMMIT
|
|
The latest updates on your projects. Learn more about Unkey Deploy
|
There was a problem hiding this comment.
1 issue found across 6 files (changes from recent commits).
Shadow auto-approve: would not auto-approve because issues were found.
Re-trigger cubic
izadoesdev
changed the title
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would require human review. Database schema changes (composite FK, migration), concurrency control (advisory lock), and new RPC endpoints with UI surface — any of these can break production data or require domain review.
Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would require human review. Contains database schema change (composite FK with cascade) and critical locking change in upsertProfile transaction; requires human review to verify migration safety and concurrency behavior.
Re-trigger cubic
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would require human review. These changes affect core data integrity and the user-facing API, including a new RPC endpoint and updated profile logic. They are high-impact and require human review.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two data-correctness fixes for the trait-history work, plus the UI/RPC surface that finally exposes traits and trait history in the product (they were stored but rendered nowhere).
Fixes (from #538 review)
profile_trait_changes.(website_id, profile_id) → profileswithON DELETE CASCADE, replacing the standalonewebsite_id → websitesref. Deleting a profile now takes its trait history with it.pg_advisory_xact_lockon the(websiteId, profileId)pair at the top of theupsertProfiletransaction (FOR UPDATEcan't lock a not-yet-existent row). Serializes concurrent first-time identifies.New: expose traits + history in the users UI
profiles.getHistoryRPC — website-read-gated, returnsprofile_trait_changesnewest-first (cap 200), each entry with changed keys (old/new) and the post-change snapshot. Two integration tests (ordering/shape + cross-org FORBIDDEN).plan/etc., hidden when empty) and a trait-history timeline (date,sourcebadge, per-keyold → newdiffs). Desktop sidebar + mobile.refetch.Deploy notes
pg_constraint:on_delete: c, 0 orphans before apply).Not addressed here
sessions.tsCTE scan expansion (pre-existing) — separate perf PR.Testing