Skip to content

Commit 5bcc758

Browse files
authored
Parse pr:modified draft→ready flips as synthetic pr:ready_for_review (#59)
- Emit pr:ready_for_review when previousDraft=true & draft=false - Skip title/description/target-only pr:modified variants - Attribute actor as author for the synthetic event - Add test fixtures and comprehensive parser tests - Update docs (README, setup guide, onboarding script)
1 parent 0fcb68d commit 5bcc758

7 files changed

Lines changed: 323 additions & 11 deletions

File tree

README.md

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,12 +63,69 @@ the data captured in v1.
6363
| **Deployment frequency** | `COUNT(*)` of `argocd_events` per `app_name` / `team` / time window where `operation_phase = 'Succeeded' AND environment = 'prod'`. Drop the `environment` filter (or slice by it) for staging visibility. |
6464
| **Lead time for changes** | For each merged PR, `MIN(bitbucket_events.occurred_at)` for the PR (first commit) → `argocd_events.occurred_at` of the prod deploy that carries the same `commit_sha` and `environment = 'prod'`. Joined via the SHA. Stratify by `bitbucket_events.change_type` (feature / hotfix / bugfix / …) to see hotfix lead time vs. feature lead time separately. |
6565
| **PR cycle time** | `pullrequest:fulfilled.occurred_at − pullrequest:created.occurred_at` per PR id. |
66-
| **Time to first review** *(DX Core 4 "code review pickup time")* | `MIN(occurred_at) WHERE event_type IN ('pr:comment:added', 'pr:reviewer:approved', 'pr:reviewer:unapproved', 'pr:reviewer:needs_work', 'pr:reviewer:updated') AND author != <pr_opener_handle> AND NOT is_automated` per PR, minus the matching `pr:opened` row. The five-event union covers every reviewer touch Bitbucket DC emits — silent approvals, retracted approvals, "needs work" flips, and bare reviewer-status changes; `NOT is_automated` strips bot comments (noergler / Renovate / etc.) — every review-time bot must have its handle in the `automation` config block, otherwise its instant comment drives the metric toward zero. |
66+
| **Time to first review** *(DX Core 4 "code review pickup time")* | Two-part computation per PR — see the SQL block below the table. Clock-start = `COALESCE(pr:ready_for_review, pr:opened)`: PRs opened ready start at `pr:opened`; PRs opened as drafts start at the synthetic `pr:ready_for_review` (emitted by the parser when a `pr:modified` payload carries `previousDraft=true, draft=false`). Engagement = first reviewer touch (`pr:comment:added`, `pr:reviewer:approved`, `pr:reviewer:unapproved`, `pr:reviewer:needs_work`, `pr:reviewer:updated`) where `author != pr_opener AND NOT is_automated AND occurred_at >= clock-start`. The five-event reviewer union covers every touch Bitbucket DC emits (silent approvals, retracted approvals, "needs work" flips, bare reviewer-status changes); the `occurred_at >= clock-start` guard drops early-feedback comments solicited during the draft phase, which would otherwise produce negative pickup times. `NOT is_automated` strips bot comments (noergler / Renovate / etc.) — every review-time bot must have its handle in the `automation` config block, otherwise its instant comment drives the metric toward zero. |
6767
| **Build success rate** | `pipeline_events` with `phase = 'COMPLETED'` grouped by `status`. Slice by `source` to compare Jenkins vs Tekton, by `pipeline_name` / `team` for ownership. |
6868
| **Build duration** | `pipeline_events.duration_seconds` (a Postgres `GENERATED ALWAYS AS (finished_at − started_at)` column). |
6969
| **Deploy success rate** | `argocd_events` with `operation_phase IN ('Succeeded', 'Failed')` aggregated, filtered to `environment = 'prod'` for the prod-only view. |
7070
| **Deploy duration** | `argocd_events.duration_seconds` (generated column). Filter by `environment = 'prod'` for production-only timing. |
7171

72+
#### Pickup-time query
73+
74+
The collector emits a synthetic `pr:ready_for_review` row when a `pr:modified`
75+
payload carries a draft→ready flip (`previousDraft=true, draft=false`); other
76+
`pr:modified` variants — title / description / target-branch changes — are
77+
dropped at parse time. The raw `eventKey` survives on `payload.eventKey` for
78+
traceability. With that in place, the metric is one CTE:
79+
80+
```sql
81+
WITH pickup_start AS (
82+
SELECT
83+
repo_full_name,
84+
pr_id,
85+
COALESCE(
86+
MIN(occurred_at) FILTER (WHERE event_type = 'pr:ready_for_review'),
87+
MIN(occurred_at) FILTER (
88+
WHERE event_type = 'pr:opened'
89+
AND COALESCE(payload->'pullRequest'->>'draft', 'false') = 'false'
90+
)
91+
) AS clock_start,
92+
MAX(author) FILTER (WHERE event_type = 'pr:opened') AS pr_opener
93+
FROM bitbucket_events
94+
GROUP BY repo_full_name, pr_id
95+
)
96+
SELECT
97+
e.repo_full_name,
98+
e.pr_id,
99+
MIN(e.occurred_at) - ps.clock_start AS pickup_interval
100+
FROM bitbucket_events e
101+
JOIN pickup_start ps USING (repo_full_name, pr_id)
102+
WHERE ps.clock_start IS NOT NULL
103+
AND e.event_type IN (
104+
'pr:comment:added',
105+
'pr:reviewer:approved',
106+
'pr:reviewer:unapproved',
107+
'pr:reviewer:needs_work',
108+
'pr:reviewer:updated'
109+
)
110+
AND e.author IS DISTINCT FROM ps.pr_opener
111+
AND NOT e.is_automated
112+
AND e.occurred_at >= ps.clock_start
113+
GROUP BY e.repo_full_name, e.pr_id, ps.clock_start;
114+
```
115+
116+
The `occurred_at >= ps.clock_start` filter is load-bearing: a reviewer can
117+
comment on a draft PR (typically when the author solicits early feedback), and
118+
without this guard the engagement timestamp could land before the ready signal
119+
and produce a negative interval. Both "early feedback in draft" and "the act of
120+
flipping the switch" are intentionally excluded from the metric — pickup time
121+
measures reviewer engagement *after the PR is ready*, nothing else.
122+
123+
The `draft = false` guard inside the `pr:opened` branch of the `COALESCE` is the
124+
same rule applied to the opposite tail: a PR opened as a draft that never gets
125+
flipped to ready has no clock-start, so it's excluded from the metric entirely.
126+
Early-feedback comments on a never-ready draft don't inflate the numerator,
127+
because there's no clock-start to subtract from in the first place.
128+
72129
### Quality / process signals from Bitbucket
73130

74131
| Metric | How it's computed |

docs/setup-bitbucket-webhook.md

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,10 +47,20 @@ The script is idempotent (rerun after edits), supports `--dry-run` and
4747
value). BBS uses this to sign each delivery.
4848
7. **Triggers** — select:
4949
- Repository: **Push**
50-
- Pull request: **Created**, **Updated**, **Approved**, **Merged**,
51-
**Declined**, **Comment created**
50+
- Pull request: **Opened**, **Source branch updated**, **Modified**,
51+
**Approved**, **Unapproved**, **Needs work**, **Reviewer updated**,
52+
**Merged**, **Deleted**, **Comment added**
5253
8. **Save**.
5354

55+
The **Modified** trigger is required for DX Core 4 pickup-time accuracy on
56+
PRs opened as drafts: BBS emits `pr:modified` on title / description /
57+
target / draft changes, and the parser keeps only the draft→ready flips
58+
(re-typed as a synthetic `pr:ready_for_review` row) — that's the
59+
clock-start signal for the pickup metric documented in the README. The
60+
**Reviewer updated** trigger covers silent reviewer-status changes
61+
(adding / removing reviewers without a comment) that also feed the
62+
pickup metric.
63+
5464
Do **not** use the `Custom headers` field for auth and do **not** populate
5565
the top-level `credentials` block via REST — BBS DC silently drops
5666
`credentials.password` on REST POST/PUT.

scripts/bitbucket_onboarding.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,10 +68,17 @@
6868
# action of any kind. A single signal (e.g. only `pr:comment:added`) is
6969
# not enough — reviewers often approve silently without typing anything,
7070
# and a comment-only signal both misses those and inflates the metric
71-
# with bot comments. The four reviewer-activity events here together
71+
# with bot comments. The five reviewer-activity events here together
7272
# cover the workflows BBS DC reviewers actually use.
73+
#
74+
# `pr:modified` is the pickup-time **start** signal, not engagement: BBS
75+
# emits it on title / description / target-branch / draft-status changes.
76+
# The parser keeps only the draft→ready flips (re-typed as a synthetic
77+
# `pr:ready_for_review`) so the pickup clock starts when the PR actually
78+
# becomes reviewable, not when it was opened as a draft.
7379
REQUIRED_WEBHOOK_EVENTS: tuple[str, ...] = (
7480
"pr:opened",
81+
"pr:modified",
7582
"pr:from_ref_updated",
7683
"pr:comment:added",
7784
"pr:reviewer:approved",

src/riptide_collector/parsers_bitbucket.py

Lines changed: 42 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,13 @@
1616
parse_change_type,
1717
)
1818

19-
# Events where the meaningful actor is the reviewer/commenter, not the
20-
# PR author. Used to power the DX Core 4 "code review pickup time" metric:
21-
# MIN(occurred_at WHERE event_type IN these AND author != pr_opener)
22-
# answers "when did someone other than the PR author engage with the PR".
19+
# Author selection only: events where `actor` is the meaningful "who did
20+
# this" — the reviewer / commenter — and `pullRequest.author` is the PR
21+
# opener. The parser uses this set to skip the pr.author lookup so the
22+
# row attributes the action to the human who performed it. The DX Core 4
23+
# pickup-time metric WHERE clause is documented in README; it overlaps
24+
# with this set but isn't derived from it (e.g. `pr:ready_for_review`
25+
# uses the same actor-as-author rule but is a clock-start, not engagement).
2326
_REVIEWER_ACTIVITY_EVENTS = frozenset(
2427
{
2528
"pr:comment:added",
@@ -30,6 +33,15 @@
3033
}
3134
)
3235

36+
# Synthetic event_type emitted by the parser when a `pr:modified` payload
37+
# carries a draft→ready flip (`previousDraft: true` + `pullRequest.draft: false`).
38+
# Re-typing at parse time keeps downstream metric queries trivial — they can
39+
# look for a row instead of digging into `payload->'previousDraft'`. The raw
40+
# eventKey is preserved on `payload.eventKey`. This is the pickup-clock
41+
# START signal for PRs that were opened as drafts; see the pickup-time
42+
# section in README for the COALESCE(opened, ready_for_review) pattern.
43+
_SYNTHETIC_READY_FOR_REVIEW = "pr:ready_for_review"
44+
3345

3446
@dataclass(frozen=True)
3547
class BitbucketEventDraft:
@@ -160,6 +172,24 @@ def extract_event(
160172
title = pr.get("title") if isinstance(pr.get("title"), str) else None
161173
description = pr.get("description") if isinstance(pr.get("description"), str) else None
162174

175+
# `pr:modified` fires on title / description / target / draft changes.
176+
# Only the draft→ready flip feeds a metric we track (DX Core 4 pickup
177+
# time start signal); other variants carry no signal worth a DB row.
178+
# Re-type the flip as `pr:ready_for_review` so downstream queries don't
179+
# have to dig into `previousDraft`; skip the rest as no-ops.
180+
if event_type == "pr:modified":
181+
previous_draft = body.get("previousDraft")
182+
current_draft = pr.get("draft")
183+
if previous_draft is True and current_draft is False:
184+
event_type = _SYNTHETIC_READY_FOR_REVIEW
185+
else:
186+
return BitbucketSkip(
187+
reason="pr:modified without draft→ready flip",
188+
delivery_id=delivery_id,
189+
event_type="pr:modified",
190+
repo_full_name=lower(raw_repo_full_name),
191+
)
192+
163193
branch_name: str | None = None
164194
commit_sha: str | None = None
165195
author: str | None = None
@@ -169,8 +199,13 @@ def extract_event(
169199
# as the meaningful "who did this" — different from pr.author who
170200
# opened the PR. We need that to attribute the "first review pickup"
171201
# signal (DX Core 4) to the right user and to filter out the
172-
# PR-author-self-comment and bot-comment noise.
173-
is_reviewer_activity = event_type in _REVIEWER_ACTIVITY_EVENTS
202+
# PR-author-self-comment and bot-comment noise. The synthetic
203+
# `pr:ready_for_review` follows the same rule: the actor is whoever
204+
# flipped the switch (often the PR author, sometimes a maintainer),
205+
# which may differ from `pullRequest.author`.
206+
is_actor_authored = (
207+
event_type in _REVIEWER_ACTIVITY_EVENTS or event_type == _SYNTHETIC_READY_FOR_REVIEW
208+
)
174209

175210
if pr:
176211
from_ref = _as_dict(pr.get("fromRef"))
@@ -180,7 +215,7 @@ def extract_event(
180215
branch_name = display_id
181216
if isinstance(latest_commit, str):
182217
commit_sha = latest_commit
183-
if not is_reviewer_activity:
218+
if not is_actor_authored:
184219
author_user = _as_dict(_as_dict(pr.get("author")).get("user"))
185220
author = _user_handle(author_user)
186221
# PR-side revert detection: the title is the only signal we have
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"eventKey": "pr:modified",
3+
"date": "2026-04-28T10:03:00+0000",
4+
"actor": {
5+
"name": "alice",
6+
"displayName": "Alice Example",
7+
"slug": "alice"
8+
},
9+
"pullRequest": {
10+
"id": 42,
11+
"title": "ABC-123: Add payment retry",
12+
"description": "Fixes ABC-123 and PROJ-9",
13+
"state": "OPEN",
14+
"draft": false,
15+
"fromRef": {
16+
"id": "refs/heads/feature/ABC-123-retries",
17+
"displayId": "feature/ABC-123-retries",
18+
"latestCommit": "abc1234567890abc1234567890abc1234567890a",
19+
"repository": {
20+
"slug": "payments-api",
21+
"project": {"key": "ACME"}
22+
}
23+
},
24+
"toRef": {
25+
"id": "refs/heads/master",
26+
"displayId": "master",
27+
"repository": {
28+
"slug": "payments-api",
29+
"project": {"key": "ACME"}
30+
}
31+
},
32+
"author": {
33+
"user": {
34+
"name": "alice",
35+
"displayName": "Alice Example",
36+
"slug": "alice"
37+
}
38+
}
39+
},
40+
"previousTitle": "ABC-123: Add payment retry",
41+
"previousDescription": "Fixes ABC-123 and PROJ-9",
42+
"previousTarget": null,
43+
"previousDraft": true
44+
}
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
{
2+
"eventKey": "pr:modified",
3+
"date": "2026-04-28T10:04:00+0000",
4+
"actor": {
5+
"name": "alice",
6+
"displayName": "Alice Example",
7+
"slug": "alice"
8+
},
9+
"pullRequest": {
10+
"id": 42,
11+
"title": "ABC-123: Add payment retry (typo fix)",
12+
"description": "Fixes ABC-123 and PROJ-9",
13+
"state": "OPEN",
14+
"draft": false,
15+
"fromRef": {
16+
"id": "refs/heads/feature/ABC-123-retries",
17+
"displayId": "feature/ABC-123-retries",
18+
"latestCommit": "abc1234567890abc1234567890abc1234567890a",
19+
"repository": {
20+
"slug": "payments-api",
21+
"project": {"key": "ACME"}
22+
}
23+
},
24+
"toRef": {
25+
"id": "refs/heads/master",
26+
"displayId": "master",
27+
"repository": {
28+
"slug": "payments-api",
29+
"project": {"key": "ACME"}
30+
}
31+
},
32+
"author": {
33+
"user": {
34+
"name": "alice",
35+
"displayName": "Alice Example",
36+
"slug": "alice"
37+
}
38+
}
39+
},
40+
"previousTitle": "ABC-123: Add payment retry",
41+
"previousDescription": "Fixes ABC-123 and PROJ-9",
42+
"previousTarget": null,
43+
"previousDraft": false
44+
}

0 commit comments

Comments
 (0)