Skip to content

Commit 3af24a7

Browse files
ci(nr): emit CI/e2e/smoke results to New Relic (Wave 5)
Wave 5 of the CI integration plan (docs/ci/01-CI-INTEGRATION-DESIGN.md §NR observability): push the web CI / Playwright / pr-smoke / e2e-prod results to New Relic so a red run — especially a user-facing-journey failure in prod — is studyable from an NR dashboard, not just the GitHub Actions log. Adds the reusable .github/actions/nr-ci-event composite action (mirrors api + worker) that POSTs an InstantCITestRun event (always) + InstantCITestFailure (on fail) to the NR Event API. Wired as if:always() steps into: - ci.yml → suite=build-and-test (tsc+vite+prerender+vitest), playwright - e2e-pr-smoke.yml → suite=pr-smoke (real-backend UI smoke; the ci-pr-smoke-failing-main P1 alert reads this) - e2e-prod.yml → suite=e2e-prod (live-prod journeys; the ci-e2e-prod-failing P1 alert reads this) The two live suites gate the emit on RUN=='1' so a fork/secret-less no-op run does NOT report a misleading pass. No-ops cleanly without the NR secret. Additive only — no gate change; npm run gate green (81 files, 1129 tests). Operator action: provision NEW_RELIC_LICENSE_KEY + NEW_RELIC_ACCOUNT_ID as GitHub Actions secrets on the instanode-web repo. NR dashboard + alerts land in the infra repo (instanode-ci-health). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 3113ae9 commit 3af24a7

4 files changed

Lines changed: 254 additions & 0 deletions

File tree

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
name: nr-ci-event
2+
description: >-
3+
Emit a CI test-run result to New Relic so a red CI run (test, e2e, smoke,
4+
deploy gate) is studyable from NR dashboards, not just GitHub logs. Posts an
5+
InstantCITestRun custom event on every gated job, plus an InstantCITestFailure
6+
event when result=fail. No-ops cleanly (logs the payload it WOULD send) when
7+
the NR secret/account is absent — never fails the calling job because NR is
8+
unreachable (fork PRs, secret not yet provisioned).
9+
10+
# Mechanism (CLAUDE.md design ref docs/ci/01-CI-INTEGRATION-DESIGN.md §NR
11+
# observability): the NR Event API is a single HTTP POST to
12+
# https://insights-collector.newrelic.com/v1/accounts/<acct>/events
13+
# authenticated with the ingest license key (the SAME NEW_RELIC_LICENSE_KEY the
14+
# Go agents use at runtime — it is a valid Insert Key for the Event API). The
15+
# account id is the numeric NEW_RELIC_ACCOUNT_ID. Both are passed as inputs from
16+
# repo secrets by the caller. When EITHER is empty the action prints the payload
17+
# and exits 0 (the no-op-without-secret contract — rule: never red a PR because
18+
# NR is down). EU-region accounts override the collector host via nr-region.
19+
20+
inputs:
21+
# --- NR credentials (caller passes from secrets; empty => no-op) ---
22+
license-key:
23+
description: >-
24+
NR ingest license key (Insert Key for the Event API). Pass
25+
${{ secrets.NEW_RELIC_LICENSE_KEY }}. Empty => action no-ops (dry-run log).
26+
required: false
27+
default: ''
28+
account-id:
29+
description: >-
30+
Numeric NR account id. Pass ${{ secrets.NEW_RELIC_ACCOUNT_ID }}.
31+
Empty => action no-ops (dry-run log).
32+
required: false
33+
default: ''
34+
nr-region:
35+
description: 'US (default) or EU — selects the insights-collector host.'
36+
required: false
37+
default: 'US'
38+
39+
# --- event payload (caller fills from the GitHub context + job result) ---
40+
result:
41+
description: 'pass | fail — usually ${{ job.status == ''success'' && ''pass'' || ''fail'' }}.'
42+
required: true
43+
suite:
44+
description: >-
45+
Logical suite name, e.g. build-and-test, coverage, playwright, pr-smoke,
46+
e2e-prod, deploy-gate. The dashboard FACETs on this.
47+
required: true
48+
repo:
49+
description: 'Repository (owner/name). Default ${{ github.repository }}.'
50+
required: false
51+
default: ''
52+
workflow:
53+
description: 'Workflow name. Default ${{ github.workflow }}.'
54+
required: false
55+
default: ''
56+
branch:
57+
description: 'Branch ref name. Default ${{ github.ref_name }}.'
58+
required: false
59+
default: ''
60+
commit-sha:
61+
description: 'Commit SHA under test. Default ${{ github.sha }}.'
62+
required: false
63+
default: ''
64+
pr-number:
65+
description: 'PR number (empty on push). Default ${{ github.event.pull_request.number }}.'
66+
required: false
67+
default: ''
68+
duration-ms:
69+
description: 'Suite duration in milliseconds (0 if not measured).'
70+
required: false
71+
default: '0'
72+
failed-step:
73+
description: 'On failure: the step/phase that failed (free text, no PII). Empty on pass.'
74+
required: false
75+
default: ''
76+
log-url:
77+
description: >-
78+
URL to the run logs for triage. Default the run's GitHub Actions URL.
79+
required: false
80+
default: ''
81+
82+
runs:
83+
using: composite
84+
steps:
85+
- name: Emit CI result to New Relic (no-op without secret)
86+
shell: bash
87+
env:
88+
# All untrusted/free-form values flow through env, never interpolated
89+
# into the shell body — same injection-safe posture as ci.yml's
90+
# dispatch-auth-contract-e2e job.
91+
NR_LICENSE_KEY: ${{ inputs.license-key }}
92+
NR_ACCOUNT_ID: ${{ inputs.account-id }}
93+
NR_REGION: ${{ inputs.nr-region }}
94+
EV_RESULT: ${{ inputs.result }}
95+
EV_SUITE: ${{ inputs.suite }}
96+
EV_REPO: ${{ inputs.repo != '' && inputs.repo || github.repository }}
97+
EV_WORKFLOW: ${{ inputs.workflow != '' && inputs.workflow || github.workflow }}
98+
EV_BRANCH: ${{ inputs.branch != '' && inputs.branch || github.ref_name }}
99+
EV_COMMIT: ${{ inputs.commit-sha != '' && inputs.commit-sha || github.sha }}
100+
EV_PR: ${{ inputs.pr-number }}
101+
EV_DURATION_MS: ${{ inputs.duration-ms }}
102+
EV_FAILED_STEP: ${{ inputs.failed-step }}
103+
EV_LOG_URL: ${{ inputs.log-url != '' && inputs.log-url || format('{0}/{1}/actions/runs/{2}', github.server_url, github.repository, github.run_id) }}
104+
EV_EVENT_NAME: ${{ github.event_name }}
105+
EV_ACTOR: ${{ github.actor }}
106+
run: |
107+
set -uo pipefail
108+
109+
# Normalise the result to the pass|fail enum the dashboard FACETs on.
110+
# Anything that isn't exactly "pass" is treated as "fail" so a typo or
111+
# a cancelled job reads as a non-pass (conservative — never a false green).
112+
case "${EV_RESULT}" in
113+
pass) RESULT="pass" ;;
114+
*) RESULT="fail" ;;
115+
esac
116+
117+
# Build the InstantCITestRun event (always) and, on fail, the
118+
# InstantCITestFailure event. jq composes the JSON so every value is
119+
# passed as an argument (no shell concatenation of free-form text).
120+
DURATION="${EV_DURATION_MS}"
121+
case "${DURATION}" in ''|*[!0-9]*) DURATION=0 ;; esac
122+
123+
RUN_EVENT=$(jq -n -c \
124+
--arg eventType "InstantCITestRun" \
125+
--arg repo "${EV_REPO}" \
126+
--arg workflow "${EV_WORKFLOW}" \
127+
--arg branch "${EV_BRANCH}" \
128+
--arg commit_sha "${EV_COMMIT}" \
129+
--arg pr_number "${EV_PR}" \
130+
--arg result "${RESULT}" \
131+
--arg suite "${EV_SUITE}" \
132+
--arg event_name "${EV_EVENT_NAME}" \
133+
--arg actor "${EV_ACTOR}" \
134+
--arg log_url "${EV_LOG_URL}" \
135+
--argjson duration_ms "${DURATION}" \
136+
'{eventType:$eventType, repo:$repo, workflow:$workflow, branch:$branch,
137+
commit_sha:$commit_sha, pr_number:$pr_number, result:$result,
138+
suite:$suite, event_name:$event_name, actor:$actor, log_url:$log_url,
139+
duration_ms:$duration_ms}')
140+
141+
PAYLOAD="[${RUN_EVENT}]"
142+
if [ "${RESULT}" = "fail" ]; then
143+
FAIL_EVENT=$(jq -n -c \
144+
--arg eventType "InstantCITestFailure" \
145+
--arg repo "${EV_REPO}" \
146+
--arg workflow "${EV_WORKFLOW}" \
147+
--arg branch "${EV_BRANCH}" \
148+
--arg commit_sha "${EV_COMMIT}" \
149+
--arg pr_number "${EV_PR}" \
150+
--arg suite "${EV_SUITE}" \
151+
--arg failed_step "${EV_FAILED_STEP}" \
152+
--arg log_url "${EV_LOG_URL}" \
153+
--arg event_name "${EV_EVENT_NAME}" \
154+
'{eventType:$eventType, repo:$repo, workflow:$workflow, branch:$branch,
155+
commit_sha:$commit_sha, pr_number:$pr_number, suite:$suite,
156+
failed_step:$failed_step, log_url:$log_url, event_name:$event_name}')
157+
PAYLOAD="[${RUN_EVENT},${FAIL_EVENT}]"
158+
fi
159+
160+
# No-op-without-secret contract: print what WOULD be sent and exit 0 so
161+
# a fork PR (no secret) or an unprovisioned repo never reds because NR
162+
# is unreachable.
163+
if [ -z "${NR_LICENSE_KEY}" ] || [ -z "${NR_ACCOUNT_ID}" ]; then
164+
echo "::notice title=nr-ci-event::NEW_RELIC_LICENSE_KEY or NEW_RELIC_ACCOUNT_ID absent — dry-run only (no event sent)."
165+
echo "would POST to NR Event API the following payload:"
166+
echo "${PAYLOAD}" | jq .
167+
exit 0
168+
fi
169+
170+
case "$(echo "${NR_REGION}" | tr '[:lower:]' '[:upper:]')" in
171+
EU) HOST="insights-collector.eu01.nr-data.net" ;;
172+
*) HOST="insights-collector.newrelic.com" ;;
173+
esac
174+
URL="https://${HOST}/v1/accounts/${NR_ACCOUNT_ID}/events"
175+
176+
echo "POSTing ${RESULT} result for suite='${EV_SUITE}' to NR account ${NR_ACCOUNT_ID} (${HOST})"
177+
HTTP_CODE=$(curl -sS -o /tmp/nr_ci_event.out -w '%{http_code}' \
178+
-X POST "${URL}" \
179+
-H "Content-Type: application/json" \
180+
-H "Api-Key: ${NR_LICENSE_KEY}" \
181+
--data-binary "${PAYLOAD}" || echo "000")
182+
183+
echo "NR Event API responded HTTP ${HTTP_CODE}"
184+
cat /tmp/nr_ci_event.out 2>/dev/null || true
185+
echo
186+
187+
# NR returns 200 on accept. Any other code (incl. network failure 000)
188+
# is logged as a warning but NEVER fails the job — observability must
189+
# not gate the pipeline.
190+
if [ "${HTTP_CODE}" != "200" ]; then
191+
echo "::warning title=nr-ci-event::NR Event API returned ${HTTP_CODE} (expected 200). CI result not recorded in NR; not failing the job."
192+
fi
193+
exit 0

.github/workflows/ci.yml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,21 @@ jobs:
5555
- run: npm run build
5656
- run: npm test
5757

58+
# Wave 5 — push the gated-test result to New Relic so a red run is
59+
# studyable from an NR dashboard, not just the GitHub Actions log.
60+
# if: always() records a FAILED tsc/build/vitest as InstantCITestRun
61+
# result=fail + InstantCITestFailure. No-ops cleanly without the NR secret.
62+
- name: Emit CI result to New Relic
63+
if: always()
64+
uses: ./.github/actions/nr-ci-event
65+
with:
66+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
67+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
68+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
69+
suite: build-and-test
70+
pr-number: ${{ github.event.pull_request.number }}
71+
failed-step: ${{ job.status != 'success' && 'gen:api-types:check / build (tsc+vite+prerender) / vitest' || '' }}
72+
5873
playwright:
5974
runs-on: ubuntu-latest
6075
steps:
@@ -68,3 +83,17 @@ jobs:
6883
- run: npm ci
6984
- run: npx playwright install --with-deps chromium
7085
- run: VITE_NO_PROXY=1 npx playwright test --project=chromium
86+
87+
# Wave 5 — record the mocked-Playwright suite outcome in NR
88+
# (suite=playwright) so a red is visible on the CI-health dashboard.
89+
# No-ops without the NR secret.
90+
- name: Emit Playwright result to New Relic
91+
if: always()
92+
uses: ./.github/actions/nr-ci-event
93+
with:
94+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
95+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
96+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
97+
suite: playwright
98+
pr-number: ${{ github.event.pull_request.number }}
99+
failed-step: ${{ job.status != 'success' && 'playwright (mocked, chromium)' || '' }}

.github/workflows/e2e-pr-smoke.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,3 +175,19 @@ jobs:
175175
e2e/.cleanup-ledger.json
176176
if-no-files-found: ignore
177177
retention-days: 7
178+
179+
# Wave 5 — record the PR-smoke outcome in NR (suite=pr-smoke). Gated on
180+
# RUN == '1' so a fork/secret-less no-op run does NOT report a misleading
181+
# pass (the suite did not actually execute). When it DID run, if: always()
182+
# captures both the journey failure and a leak-reaper failure. No-ops
183+
# without the NR secret. The pr-smoke-failing-main NR alert reads this.
184+
- name: Emit PR-smoke result to New Relic
185+
if: always() && env.RUN == '1'
186+
uses: ./.github/actions/nr-ci-event
187+
with:
188+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
189+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
190+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
191+
suite: pr-smoke
192+
pr-number: ${{ github.event.pull_request.number }}
193+
failed-step: ${{ job.status != 'success' && 'real-backend UI @pr-smoke journeys / mint / reap' || '' }}

.github/workflows/e2e-prod.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,3 +196,19 @@ jobs:
196196
e2e/.cleanup-ledger.json
197197
if-no-files-found: ignore
198198
retention-days: 14
199+
200+
# Wave 5 — record the live-prod E2E outcome in NR (suite=e2e-prod). This
201+
# is the user-facing-journeys-against-prod signal; the e2e-prod-suite-
202+
# failing NR alert (P1) reads it. Gated on RUN == '1' so a secret-less
203+
# no-op run does NOT report a misleading pass. if: always() captures the
204+
# journey failure and the leak-reaper failure. No-ops without the NR
205+
# secret.
206+
- name: Emit e2e-prod result to New Relic
207+
if: always() && env.RUN == '1'
208+
uses: ./.github/actions/nr-ci-event
209+
with:
210+
license-key: ${{ secrets.NEW_RELIC_LICENSE_KEY }}
211+
account-id: ${{ secrets.NEW_RELIC_ACCOUNT_ID }}
212+
result: ${{ job.status == 'success' && 'pass' || 'fail' }}
213+
suite: e2e-prod
214+
failed-step: ${{ job.status != 'success' && 'live-prod E2E journeys / mint / reap' || '' }}

0 commit comments

Comments
 (0)