Merge pull request #45 from trick77/feat/argocd-images-field

feat(argocd): require images[] as bridge to pipeline SHA

Author: trick77

AGENTS.md

@@ -26,11 +26,11 @@ If `docker ps` fails, ask the user to start OrbStack.
 - **Raw payload always stored.** `payload JSONB` keeps the full request body even if fields are extracted into typed columns. Don't drop fields you don't currently use.
 - **`riptide.json` is config, not data.** `openshift/collector/riptide.json` is the in-repo sample; it declares teams (name + `group_email`) and org-wide automation rules. Edits go through PRs. The running pod hot-reloads via mtime in `RiptideConfigStore.maybe_reload()`. Do not propose moving the config into Postgres.
 - **Per-team bearer keys live in a separate file**: in production it is mounted from the `riptide-collector-team-keys` Secret (never committed); the in-repo `openshift/collector/team-keys.json` is a dev sample with deterministic test hashes (raw dev bearers documented in `compose.yaml`). Stored as sha256 hashes; `TeamKeysStore` hot-reloads it the same way as the config. The bearer **is** the team identity — every webhook is tagged with `team = caller_team`.
-- **No `service` column. No `service_id` on the wire.** Per-source aggregations group by `repo_full_name` / `pipeline_name` / `app_name` / `repo`; cross-source aggregations join on `commit_sha`; org-wide rollups group by `team`. Identifiers are lowercased at ingest (`commit_sha`, `revision`, `repo_full_name`, `branch_name`, `repo`) so SQL joins are case-stable. Do not propose adding a unified `service` column or `service_id` field — it served only single-pane labelling and was dropped.
+- **No `service` column. No `service_id` on the wire.** Per-source aggregations group by `repo_full_name` / `pipeline_name` / `app_name` / `repo`; org-wide rollups group by `team`. Cross-source joins for BB↔Pipeline use `commit_sha`; Argo CD joins are described in the next bullet. Identifiers are lowercased at ingest (`commit_sha`, `revision`, `repo_full_name`, `branch_name`, `repo`) so SQL joins are case-stable. Do not propose adding a unified `service` column or `service_id` field — it served only single-pane labelling and was dropped.
 - **`automation` is org-wide.** Bot definitions live at the config root, not per team.
 - **Metrics are computed on read, not at ingest.** Don't add aggregation tables or scheduled rollup jobs in v1. Schema additions should preserve raw events; new metrics are SQL queries against existing rows or future materialized views.
-- **Commit SHA is the universal join key.** All three sources record `commit_sha`/`revision`. Lead-time joins between `bitbucket_events`, `pipeline_events`, `argocd_events` happen on this column.
-- **`change_type` lives on Bitbucket events only.** Don't denormalise it onto pipeline / Argo rows; join via `commit_sha` at read time.
+- **Commit SHA joins Bitbucket↔Pipeline; Argo CD needs `payload->'images'`.** `bitbucket_events.commit_sha = pipeline_events.commit_sha` is a deterministic join (App-repo SHA on both sides). `argocd_events.revision` is the **GitOps-repo SHA** (proven empirically: four Apps for one service share one revision), so it does NOT directly match the other two. The App-repo SHA is embedded in image tags rendered via `.app.status.summary.images`; the receiver stores them in `payload->'images'`. A future correlator extracts SHAs from those tags to bridge Argo CD events to pipeline events. Do not propose adding a `service_id` or hand-coded service-name mappings to fix correlation — the image-tag SHA is the contract.
+- **`change_type` lives on Bitbucket events only.** Don't denormalise it onto pipeline / Argo rows; join Pipeline rows via `commit_sha` and Argo rows via the image-tag SHA in `payload->'images'` at read time.
 - **CI events are source-tagged, not source-routed.** All pipeline events from any CI (Jenkins, Tekton, …) land in the single `pipeline_events` table via `POST /webhooks/pipeline`, distinguished by the `source` column. Do not add per-CI tables or endpoints. The dedup key is `source#pipeline_name#run_id#phase`.
 - **Noergler events carry finops + reviewer-precision only.** The `noergler_events` table is `event_type`-discriminated (`completed` | `feedback`) and is fed by `POST /webhooks/noergler` from optional noergler instances. Do not re-emit PR lifecycle from noergler — `bitbucket_events` already covers open / merged / declined. Dedup keys: `completed#<run_id>` and `feedback#<finding_id>#<verdict>`.
 - **Senders verify reachability + bearer at startup via `GET /auth/ping`.** Authenticated endpoint returning `{"status":"ok","team":"<caller_team>"}`. Use this from any sender (noergler, future ones) to fail-fast on a wrong token. Don't reuse `/health` (unauth liveness) or `/ready` (unauth readiness) for this — those answer different questions.

docs/argocd-notification-template.yaml

@@ -8,6 +8,14 @@ data:
     webhook:
       riptide:
         method: POST
+        # `toJson` / `default` / `list` are sprig functions (notifications-engine
+        # registers sprig.TxtFuncMap in pkg/templates/service.go), so they're
+        # available here. `toJson` serialises the image slice as a valid JSON
+        # array and escapes any quote/backslash inside tag strings; a manual
+        # {{range}} loop would emit invalid JSON if a tag ever contained a `"`.
+        # `default (list) ...` collapses a nil summary into `[]` so the body
+        # is always valid JSON, even before Argo has populated the summary
+        # (which would otherwise render as `null` and fail the receiver schema).
         body: |
           {
             "app_name": "{{.app.metadata.name}}",
@@ -16,7 +24,8 @@ data:
             "operation_phase": "{{.app.status.operationState.phase}}",
             "started_at": "{{.app.status.operationState.startedAt}}",
             "finished_at": "{{.app.status.operationState.finishedAt}}",
-            "destination_namespace": "{{.app.spec.destination.namespace}}"
+            "destination_namespace": "{{.app.spec.destination.namespace}}",
+            "images": {{toJson (default (list) .app.status.summary.images)}}
           }
 
   trigger.on-deployed: |

docs/setup-argocd-notification.md

@@ -91,14 +91,25 @@ oc -n argocd apply -f docs/argocd-notification-template.yaml
 ```
 
 This adds:
-- `template.app-deployed-riptide`
+- `template.app-deployed-riptide` — the webhook body includes `app_name`,
+  `revision`, `sync_status`, `operation_phase`, `started_at`, `finished_at`,
+  `destination_namespace`, and `images` (a JSON array rendered from
+  `.app.status.summary.images`). `images` is the bridge for joining Argo CD
+  events to pipeline events: `revision` is the GitOps-repo SHA, but image
+  tags typically embed the App-repo commit SHA that the pipeline reports.
 - `trigger.on-deployed`, `trigger.on-sync-succeeded`, `trigger.on-sync-failed`
   (riptide-flavored)
 
-If you are upgrading from an earlier riptide release, **re-apply this
-ConfigMap** so the template body includes the new `destination_namespace`
-field — riptide derives the `environment` column (and the prod-vs-non-prod
-metric filters) from the namespace suffix.
+> **Required field, hard cutover.** `images` is required on the receiver
+> side — webhooks rendered by an outdated ConfigMap will be rejected with
+> HTTP 422. Apply this ConfigMap **before** rolling out a receiver that
+> expects `images`, then restart the notifications controller so it
+> re-reads the template:
+> ```bash
+> oc -n argocd apply -f docs/argocd-notification-template.yaml
+> oc -n argocd rollout restart deploy/argocd-notifications-controller
+> ```
+> Reversing the order strands events between deploy and ConfigMap-apply.
 
 ## 3) Route teams to their service via AppProject defaults
 
@@ -218,5 +229,17 @@ suffix counts as "production" is configured in
 default `prod`). To keep the database small, list non-prod stage suffixes
 in `environments.ignored_stages` (e.g. `["dev", "entw", "syst", "stage"]`)
 — matching events return `202 {"status":"ignored"}` and are dropped before
-insert. Aggregations group by `app_name`; cross-source joins (Pipeline,
-Bitbucket, Noergler) use `revision = commit_sha`.
+insert. Aggregations group by `app_name`.
+
+The rendered image list lives in `payload->'images'`:
+
+```sql
+SELECT app_name, payload->'images' AS images FROM argocd_events
+ORDER BY created_at DESC LIMIT 5;
+```
+
+`revision` is the **GitOps-repo SHA**, not the App-repo SHA — direct joins
+to `pipeline_events.commit_sha` or `bitbucket_events.commit_sha` will not
+match. The App-repo SHA is typically embedded in the image tag (e.g.
+`registry/app:abc1234`); a future reader/correlator pulls SHAs out of
+`payload->'images'` to bridge to pipeline events.

src/riptide_collector/schemas/argocd.py

@@ -18,6 +18,16 @@ class ArgoCDWebhook(BaseModel):
         default=None,
         description="kubernetes namespace the app deployed into; suffix drives `environment`",
     )
+    images: list[str] = Field(
+        ...,
+        description=(
+            "rendered images from the synced manifests (Argo CD field path "
+            "`.app.status.summary.images`). may be empty for apps without pods. "
+            "the App-Repo commit SHA is typically embedded in the image tag and "
+            "bridges argocd.revision (GitOps-repo SHA) to pipeline.commit_sha "
+            "(App-repo SHA) — correlation logic is built on top of this field."
+        ),
+    )
 
     @field_validator("started_at", "finished_at")
     @classmethod

tests/fixtures/argocd_synced.json

@@ -5,5 +5,8 @@
   "operation_phase": "Succeeded",
   "started_at": "2026-04-28T10:09:00Z",
   "finished_at": "2026-04-28T10:09:45Z",
-  "destination_namespace": "payments-prod"
+  "destination_namespace": "payments-prod",
+  "images": [
+    "quay.io/team/payments-api:abc1234"
+  ]
 }

tests/test_auth.py

@@ -133,6 +133,7 @@ async def test_argocd_token_accepted_on_argocd(self, client: AsyncClient) -> Non
             json={
                 "app_name": "x",
                 "revision": "abc1234567890abc1234567890abc1234567890a",
+                "images": [],
             },
             headers=_bearer(CHECKOUT_ARGOCD),
         )

tests/test_webhooks.py

@@ -320,6 +320,40 @@ async def test_missing_revision_returns_422(self, client: AsyncClient) -> None:
         response = await client.post("/webhooks/argocd", json=payload, headers=ARGOCD_AUTH)
         assert response.status_code == 422
 
+    async def test_missing_images_returns_422(self, client: AsyncClient) -> None:
+        payload = _load("argocd_synced.json")
+        del payload["images"]
+        response = await client.post("/webhooks/argocd", json=payload, headers=ARGOCD_AUTH)
+        assert response.status_code == 422
+
+    async def test_empty_images_list_accepted(self, client: AsyncClient) -> None:
+        payload = _load("argocd_synced.json")
+        payload["images"] = []
+        response = await client.post("/webhooks/argocd", json=payload, headers=ARGOCD_AUTH)
+        assert response.status_code == 202
+
+        factory = TestBitbucketWebhook._fresh_session_factory(client)
+        async with factory() as session:
+            row = (await session.execute(select(ArgoCDEvent))).scalar_one()
+            assert row.payload["images"] == []
+
+    async def test_images_persisted_in_payload(self, client: AsyncClient) -> None:
+        payload = _load("argocd_synced.json")
+        payload["images"] = [
+            "quay.io/team/payments-api:abc1234",
+            "quay.io/team/payments-worker:abc1234",
+        ]
+        response = await client.post("/webhooks/argocd", json=payload, headers=ARGOCD_AUTH)
+        assert response.status_code == 202
+
+        factory = TestBitbucketWebhook._fresh_session_factory(client)
+        async with factory() as session:
+            row = (await session.execute(select(ArgoCDEvent))).scalar_one()
+            assert row.payload["images"] == [
+                "quay.io/team/payments-api:abc1234",
+                "quay.io/team/payments-worker:abc1234",
+            ]
+
     async def test_ignored_stage_is_dropped(self, client_with_ignored_stages: AsyncClient) -> None:
         payload = _load("argocd_synced.json")
         payload["destination_namespace"] = "checkout-dev"

tests/test_webhooks_extra.py

@@ -78,6 +78,7 @@ async def test_argocd_phase_transitions_create_distinct_rows(
         "revision": "abc1234567890abc1234567890abc1234567890a",
         "sync_status": "Synced",
         "started_at": "2026-04-28T10:09:00Z",
+        "images": ["quay.io/team/payments-api:abc1234"],
     }
     running = {**base, "operation_phase": "Running"}
     succeeded = {**base, "operation_phase": "Succeeded", "finished_at": "2026-04-28T10:09:45Z"}