Skip to content

Commit 9a508db

Browse files
mastermanas805claude
mastermanas805
and
claude
authored
feat(synthetic): money/value-journey legs + CI→NR emit (Wave 5) (#95)
* feat(synthetic): money/value-journey legs + CI→NR emit (Wave 5) Wave 5 of the CI integration plan (docs/ci/01-CI-INTEGRATION-DESIGN.md). Two additive pieces, both flag/secret-gated and inert by default. 1) Generalize the prod synthetic monitor to the money/value journeys. flow_synthetic_money.go adds four legs to the continuous flow matrix, each reading a TRUTH SURFACE (CLAUDE.md rule 12), run by the SAME 5-min flow_synthetic River job under the SAME FLOW_SYNTHETIC_ENABLED master flag + per-flow FLOW_SYNTHETIC_DISABLED kill list, emitting the SAME instant_flow_test_* counters + InstantFlowTest event (cohort=synthetic): - claim — GET /claim/preview invalid-token → assert 400 invalid_token contract (anon→claimed funnel step; single-use-preserving). - deploy_status — POST /deploy/new → assert the tier-gate 402 wall on a no-headroom tier (or 201 + reap on a headroom tier, rule 24). - checkout — POST /api/v1/billing/checkout → assert non-5xx reachability (contract-only; Razorpay live recurring operator-blocked). - magic_link — POST /auth/email/start (202) THEN read forwarder_sent .classification (rule 12 — a 'rejected' classification, the Brevo-unvalidated-sender reality, FAILS the leg; the 202 alone is NOT a delivery). Work() mints the session JWT once and shares it across the P0 + money matrices. flow_synthetic_money_test.go + extended flow_synthetic_test.go give 100% patch coverage of the new file (every contract/error/degraded branch, the headroom deploy create→reap + reap-leak/transport-error paths, the classifier seam). Inert when FLOW_SYNTHETIC_ENABLED is unset (the flag-off no-op test proves it). 2) CI test-run → New Relic emit (reusable composite action). .github/actions/nr-ci-event POSTs InstantCITestRun (always) + InstantCITestFailure (on fail) to the NR Event API; wired into ci.yml build-and-test + integration as an if:always() step. No-ops cleanly without NEW_RELIC_LICENSE_KEY/ACCOUNT_ID. Per-package `go test ./internal/jobs/ -short` GREEN; the flow tests pass deterministically. NOTE: the full `go test ./... -p 1` run has a PRE-EXISTING intermittent flake in the jobs package (River "PeriodicJobEnqueuer: context canceled" teardown race, ~1/3 runs, unrelated to these hermetic httptest+sqlmock flow tests) — per memory feedback_coverage_measure_per_package_not_dotdotdot the authoritative gate is CI. Operator action: provision NEW_RELIC_LICENSE_KEY + NEW_RELIC_ACCOUNT_ID as GitHub secrets on the worker repo; enable the synthetic suite per infra/FLOW-SYNTHETIC-RUNBOOK.md (FLOW_SYNTHETIC_ENABLED=true). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): composite action cannot read github/secrets/job — caller passes them as inputs The nr-ci-event composite action referenced github.*/secrets.*/job.status in its own env: block; GitHub rejects those inside a composite action. Move resolution to the caller's with: block; action reads only inputs.*. Adds event-name + actor inputs; callers pass repo/workflow/branch/commit-sha/log-url/event-name/actor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): strip ${{ }} from composite-action input descriptions GitHub evaluates ${{ }} expressions even inside an input's description: string, so the example 'Pass ${{ secrets.NEW_RELIC_LICENSE_KEY }}' text in the descriptions triggered the same Unrecognized-named-value validation error. Use plain backtick-quoted text in all descriptions; only the runs: body keeps inputs.* expressions (the one context a composite action may read). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 91ef958 commit 9a508db

6 files changed

Lines changed: 1314 additions & 36 deletions

File tree

‎.github/actions/nr-ci-event/action.yml‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
name: nr-ci-event
2+
description: >-
3+
Emit a CI test-run result to New Relic so a red CI run (test, e2e, smoke,
4+
deploy gate) is studyable from NR dashboards, not just GitHub logs. Posts an
5+
InstantCITestRun custom event on every gated job, plus an InstantCITestFailure
6+
event when result=fail. No-ops cleanly (logs the payload it WOULD send) when
7+
the NR secret/account is absent — never fails the calling job because NR is
8+
unreachable (fork PRs, secret not yet provisioned).
9+
10+
# Mechanism (CLAUDE.md design ref docs/ci/01-CI-INTEGRATION-DESIGN.md §NR
11+
# observability): the NR Event API is a single HTTP POST to
12+
# https://insights-collector.newrelic.com/v1/accounts/<acct>/events
13+
# authenticated with the ingest license key (the SAME NEW_RELIC_LICENSE_KEY the
14+
# Go agents use at runtime — it is a valid Insert Key for the Event API). The
15+
# account id is the numeric NEW_RELIC_ACCOUNT_ID. Both are passed as inputs from
16+
# repo secrets by the caller. When EITHER is empty the action prints the payload
17+
# and exits 0 (the no-op-without-secret contract — rule: never red a PR because
18+
# NR is down). EU-region accounts override the collector host via nr-region.
19+
20+
inputs:
21+
# --- NR credentials (caller passes from secrets; empty => no-op) ---
22+
license-key:
23+
description: >-
24+
NR ingest license key (Insert Key for the Event API). Pass
25+
`secrets.NEW_RELIC_LICENSE_KEY`. Empty => action no-ops (dry-run log).
26+
required: false
27+
default: ''
28+
account-id:
29+
description: >-
30+
Numeric NR account id. Pass `secrets.NEW_RELIC_ACCOUNT_ID`.
31+
Empty => action no-ops (dry-run log).
32+
required: false
33+
default: ''
34+
nr-region:
35+
description: 'US (default) or EU — selects the insights-collector host.'
36+
required: false
37+
default: 'US'
38+
39+
# --- event payload (caller fills from the GitHub context + job result) ---
40+
result:
41+
description: 'pass | fail — usually `job.status == ''success'' && ''pass'' || ''fail''`.'
42+
required: true
43+
suite:
44+
description: >-
45+
Logical suite name, e.g. build-and-test, coverage, playwright, pr-smoke,
46+
e2e-prod, deploy-gate. The dashboard FACETs on this.
47+
required: true
48+
# NOTE: composite actions cannot read the `github` context in their own
49+
# expressions, so the caller MUST pass these from its `with:` block (e.g.
50+
# repo: `github.repository`). The defaults below only apply when a caller
51+
# omits them entirely.
52+
repo:
53+
description: 'Repository (owner/name). Caller passes `github.repository`.'
54+
required: false
55+
default: ''
56+
workflow:
57+
description: 'Workflow name. Caller passes `github.workflow`.'
58+
required: false
59+
default: ''
60+
branch:
61+
description: 'Branch ref name. Caller passes `github.ref_name`.'
62+
required: false
63+
default: ''
64+
commit-sha:
65+
description: 'Commit SHA under test. Caller passes `github.sha`.'
66+
required: false
67+
default: ''
68+
pr-number:
69+
description: 'PR number (empty on push). Caller passes `github.event.pull_request.number`.'
70+
required: false
71+
default: ''
72+
duration-ms:
73+
description: 'Suite duration in milliseconds (0 if not measured).'
74+
required: false
75+
default: '0'
76+
failed-step:
77+
description: 'On failure: the step/phase that failed (free text, no PII). Empty on pass.'
78+
required: false
79+
default: ''
80+
log-url:
81+
description: 'URL to the run logs for triage. Caller passes the run URL.'
82+
required: false
83+
default: ''
84+
event-name:
85+
description: 'GitHub event name. Caller passes `github.event_name`.'
86+
required: false
87+
default: ''
88+
actor:
89+
description: 'GitHub actor. Caller passes `github.actor`.'
90+
required: false
91+
default: ''
92+
93+
runs:
94+
using: composite
95+
steps:
96+
- name: Emit CI result to New Relic (no-op without secret)
97+
shell: bash
98+
env:
99+
# Composite actions may reference ONLY `inputs` (+ env/runner/steps) in
100+
# their expressions — `secrets`, `job`, and `github` are NOT available
101+
# here, so the CALLER resolves those and passes them as inputs. All
102+
# untrusted/free-form values flow through env, never interpolated into
103+
# the shell body (injection-safe — same posture as ci.yml's
104+
# dispatch-auth-contract-e2e job).
105+
NR_LICENSE_KEY: ${{ inputs.license-key }}
106+
NR_ACCOUNT_ID: ${{ inputs.account-id }}
107+
NR_REGION: ${{ inputs.nr-region }}
108+
EV_RESULT: ${{ inputs.result }}
109+
EV_SUITE: ${{ inputs.suite }}
110+
EV_REPO: ${{ inputs.repo }}
111+
EV_WORKFLOW: ${{ inputs.workflow }}
112+
EV_BRANCH: ${{ inputs.branch }}
113+
EV_COMMIT: ${{ inputs.commit-sha }}
114+
EV_PR: ${{ inputs.pr-number }}
115+
EV_DURATION_MS: ${{ inputs.duration-ms }}
116+
EV_FAILED_STEP: ${{ inputs.failed-step }}
117+
EV_LOG_URL: ${{ inputs.log-url }}
118+
EV_EVENT_NAME: ${{ inputs.event-name }}
119+
EV_ACTOR: ${{ inputs.actor }}
120+
run: |
121+
set -uo pipefail
122+
123+
# Normalise the result to the pass|fail enum the dashboard FACETs on.
124+
# Anything that isn't exactly "pass" is treated as "fail" so a typo or
125+
# a cancelled job reads as a non-pass (conservative — never a false green).
126+
case "${EV_RESULT}" in
127+
pass) RESULT="pass" ;;
128+
*) RESULT="fail" ;;
129+
esac
130+
131+
# Build the InstantCITestRun event (always) and, on fail, the
132+
# InstantCITestFailure event. jq composes the JSON so every value is
133+
# passed as an argument (no shell concatenation of free-form text).
134+
DURATION="${EV_DURATION_MS}"
135+
case "${DURATION}" in ''|*[!0-9]*) DURATION=0 ;; esac
136+
137+
RUN_EVENT=$(jq -n -c \
138+
--arg eventType "InstantCITestRun" \
139+
--arg repo "${EV_REPO}" \
140+
--arg workflow "${EV_WORKFLOW}" \
141+
--arg branch "${EV_BRANCH}" \
142+
--arg commit_sha "${EV_COMMIT}" \
143+
--arg pr_number "${EV_PR}" \
144+
--arg result "${RESULT}" \
145+
--arg suite "${EV_SUITE}" \
146+
--arg event_name "${EV_EVENT_NAME}" \
147+
--arg actor "${EV_ACTOR}" \
148+
--arg log_url "${EV_LOG_URL}" \
149+
--argjson duration_ms "${DURATION}" \
150+
'{eventType:$eventType, repo:$repo, workflow:$workflow, branch:$branch,
151+
commit_sha:$commit_sha, pr_number:$pr_number, result:$result,
152+
suite:$suite, event_name:$event_name, actor:$actor, log_url:$log_url,
153+
duration_ms:$duration_ms}')
154+
155+
PAYLOAD="[${RUN_EVENT}]"
156+
if [ "${RESULT}" = "fail" ]; then
157+
FAIL_EVENT=$(jq -n -c \
158+
--arg eventType "InstantCITestFailure" \
159+
--arg repo "${EV_REPO}" \
160+
--arg workflow "${EV_WORKFLOW}" \
161+
--arg branch "${EV_BRANCH}" \
162+
--arg commit_sha "${EV_COMMIT}" \
163+
--arg pr_number "${EV_PR}" \
164+
--arg suite "${EV_SUITE}" \
165+
--arg failed_step "${EV_FAILED_STEP}" \
166+
--arg log_url "${EV_LOG_URL}" \
167+
--arg event_name "${EV_EVENT_NAME}" \
168+
'{eventType:$eventType, repo:$repo, workflow:$workflow, branch:$branch,
169+
commit_sha:$commit_sha, pr_number:$pr_number, suite:$suite,
170+
failed_step:$failed_step, log_url:$log_url, event_name:$event_name}')
171+
PAYLOAD="[${RUN_EVENT},${FAIL_EVENT}]"
172+
fi
173+
174+
# No-op-without-secret contract: print what WOULD be sent and exit 0 so
175+
# a fork PR (no secret) or an unprovisioned repo never reds because NR
176+
# is unreachable.
177+
if [ -z "${NR_LICENSE_KEY}" ] || [ -z "${NR_ACCOUNT_ID}" ]; then
178+
echo "::notice title=nr-ci-event::NEW_RELIC_LICENSE_KEY or NEW_RELIC_ACCOUNT_ID absent — dry-run only (no event sent)."
179+
echo "would POST to NR Event API the following payload:"
180+
echo "${PAYLOAD}" | jq .
181+
exit 0
182+
fi
183+
184+
case "$(echo "${NR_REGION}" | tr '[:lower:]' '[:upper:]')" in
185+
EU) HOST="insights-collector.eu01.nr-data.net" ;;
186+
*) HOST="insights-collector.newrelic.com" ;;
187+
esac
188+
URL="https://${HOST}/v1/accounts/${NR_ACCOUNT_ID}/events"
189+
190+
echo "POSTing ${RESULT} result for suite='${EV_SUITE}' to NR account ${NR_ACCOUNT_ID} (${HOST})"
191+
HTTP_CODE=$(curl -sS -o /tmp/nr_ci_event.out -w '%{http_code}' \
192+
-X POST "${URL}" \
193+
-H "Content-Type: application/json" \
194+
-H "Api-Key: ${NR_LICENSE_KEY}" \
195+
--data-binary "${PAYLOAD}" || echo "000")
196+
197+
echo "NR Event API responded HTTP ${HTTP_CODE}"
198+
cat /tmp/nr_ci_event.out 2>/dev/null || true
199+
echo
200+
201+
# NR returns 200 on accept. Any other code (incl. network failure 000)
202+
# is logged as a warning but NEVER fails the job — observability must
203+
# not gate the pipeline.
204+
if [ "${HTTP_CODE}" != "200" ]; then
205+
echo "::warning title=nr-ci-event::NR Event API returned ${HTTP_CODE} (expected 200). CI result not recorded in NR; not failing the job."
206+
fi
207+
exit 0

‎.github/workflows/ci.yml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,28 @@ jobs:
9999
INSTANT_API_REPO: ${{ github.workspace }}/../api
100100
run: go test ./... -v -race -count=1
101101

102+
# Wave 5 — push the gated-test result to New Relic so a red run is
103+
# studyable from an NR dashboard, not just the GitHub Actions log.
104+
# if: always() records a FAILED `go test` as InstantCITestRun result=fail
105+
# + InstantCITestFailure. No-ops cleanly without the NR secret.
106+
- name: Emit CI result to New Relic
107+
if: always()
108+
uses: ./.github/actions/nr-ci-event
109+
with:
110+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
111+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
112+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
113+
suite: build-and-test
114+
pr-number: ${{ github.event.pull_request.number }}
115+
failed-step: ${{ job.status != 'success' && 'go build / vet / test (-race)' || '' }}
116+
repo: ${{ github.repository }}
117+
workflow: ${{ github.workflow }}
118+
branch: ${{ github.ref_name }}
119+
commit-sha: ${{ github.sha }}
120+
log-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
121+
event-name: ${{ github.event_name }}
122+
actor: ${{ github.actor }}
123+
102124
# ------------------------------------------------------------------
103125
# GATING integration job (W2-T1).
104126
#
@@ -212,3 +234,24 @@ jobs:
212234
env:
213235
INSTANT_API_REPO: ${{ github.workspace }}/../api
214236
run: go test ./internal/... -run Integration -count=1 -p 1 -v
237+
238+
# Wave 5 — record the real-DB integration-gate outcome in NR
239+
# (suite=integration) so a red here is visible alongside the unit gate on
240+
# the CI-health dashboard. No-ops without the NR secret.
241+
- name: Emit integration result to New Relic
242+
if: always()
243+
uses: ./.github/actions/nr-ci-event
244+
with:
245+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
246+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
247+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
248+
suite: integration
249+
pr-number: ${{ github.event.pull_request.number }}
250+
failed-step: ${{ job.status != 'success' && 'real-DB integration tests' || '' }}
251+
repo: ${{ github.repository }}
252+
workflow: ${{ github.workflow }}
253+
branch: ${{ github.ref_name }}
254+
commit-sha: ${{ github.sha }}
255+
log-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
256+
event-name: ${{ github.event_name }}
257+
actor: ${{ github.actor }}

‎internal/jobs/flow_synthetic.go‎

Lines changed: 34 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -375,7 +375,28 @@ func (w *FlowSyntheticWorker) Work(ctx context.Context, job *river.Job[FlowSynth
375375
// flows (recorded per-flow below).
376376
seedOK := w.ensureSyntheticTeam(ctx)
377377

378-
results := w.runMatrix(ctx, runID, commitID, seedOK)
378+
// Mint the session JWT once for every authed leg (P0 + money). If the seed
379+
// failed OR the mint fails, the authed legs degrade (config/DB drift, not an
380+
// outage) — recorded per-flow inside each matrix.
381+
bearer := ""
382+
if seedOK {
383+
var err error
384+
bearer, err = w.mintSessionJWT()
385+
if err != nil {
386+
seedOK = false
387+
slog.Warn("jobs.flow_synthetic.mint_failed", "reason", err.Error())
388+
}
389+
}
390+
391+
results := w.runMatrix(ctx, runID, commitID, bearer, seedOK)
392+
393+
// Money/value-journey legs (claim, deploy gate, checkout, magic-link →
394+
// Brevo classification) — the value paths a paying customer's money rides
395+
// on, each reading a truth surface (rule 12). Same flag, same counters/event,
396+
// same matrix. See flow_synthetic_money.go.
397+
for flow, res := range w.runMoneyMatrix(ctx, runID, commitID, bearer, seedOK) {
398+
results[flow] = res
399+
}
379400

380401
// Reaper backstop: sweep any synthetic resource an earlier mid-leg crash
381402
// left behind. Inline reap already handles the happy path; this catches
@@ -408,26 +429,17 @@ type flowSyntheticResult struct {
408429
// runMatrix executes every (enabled) flow in the P0 set sequentially and
409430
// returns a flow→result map for the completion log. Each flow is wrapped by
410431
// runFlow (timeout + recover + record).
411-
func (w *FlowSyntheticWorker) runMatrix(ctx context.Context, runID, commitID string, seedOK bool) map[string]string {
432+
func (w *FlowSyntheticWorker) runMatrix(ctx context.Context, runID, commitID, bearer string, seedOK bool) map[string]string {
412433
out := map[string]string{}
413434

414435
// Flow 1 — healthz (anonymous, no auth).
415436
out[flowHealthz] = w.runFlow(ctx, runID, commitID, flowHealthz, func(fctx context.Context) flowSyntheticResult {
416437
return w.flowHealthz(fctx)
417438
})
418439

419-
// Flows 2 & 3 require the seeded team + a minted session JWT. If the seed
420-
// failed they degrade (config/DB drift, not an outage) so they don't page.
421-
bearer := ""
422-
if seedOK {
423-
var err error
424-
bearer, err = w.mintSessionJWT()
425-
if err != nil {
426-
seedOK = false
427-
slog.Warn("jobs.flow_synthetic.mint_failed", "reason", err.Error())
428-
}
429-
}
430-
440+
// Flows 2 & 3 require the seeded team + the minted session JWT (minted once
441+
// in Work and passed in). If the seed failed / mint failed seedOK is false
442+
// and they degrade (config/DB drift, not an outage) so they don't page.
431443
out[flowAuthMe] = w.runFlow(ctx, runID, commitID, flowAuthMe, func(fctx context.Context) flowSyntheticResult {
432444
if !seedOK {
433445
return flowSyntheticResult{flow: flowAuthMe, actor: flowActorHuman, tier: w.cfg.Tier, result: flowResultDegraded, reason: "synthetic team/JWT unavailable — flow skipped"}
@@ -489,6 +501,14 @@ func flowActorForFlow(flow string) string {
489501
return flowActorHuman
490502
case flowProvisionReap:
491503
return flowActorAgent
504+
case flowClaim:
505+
return flowActorAnon
506+
case flowMagicLink:
507+
return flowActorHuman
508+
case flowCheckout:
509+
return flowActorHuman
510+
case flowDeployStatus:
511+
return flowActorAgent
492512
default:
493513
return analyticsevent.ActorUnknown
494514
}

0 commit comments

Comments
 (0)