Skip to content

Commit e0faa08

Browse files
connortsui20connortsui20
authored
ci: fix v4 ingest uv/sccache ordering; rename commit-metadata workflow (#8516)
Two related cleanups to the v4 emitter dual-write workflows. **Fix the v4 ingest sccache failure.** The v4 Postgres dual-write step assumes `GitHubBenchmarkIngestRole` via `configure-aws-credentials`, which persists those credentials as the job's ambient AWS credentials. The ingest role only has `rds-db:connect`, so the immediately-following "Install uv" step — which runs `uv sync` and compiles `vortex-python` through sccache (S3-backed) — intermittently failed with `s3:GetObject AccessDenied` on the benchmark runners (the ones with `extras=s3-cache`). The step is `continue-on-error`, so CI stayed green, but the affected jobs' v4 writes silently dropped. This reorders the v4 block so "Install uv" runs before "Configure AWS credentials": sccache keeps the job's original S3-capable credentials, and the role is assumed immediately before the ingest, which needs only `rds-db:connect`. **Rename the commit-metadata workflow.** `v3-commit-metadata.yml` uploads the `commits` dimension to both the v3 server and the v4 Postgres, so naming it "v3" is misleading — renamed to `commit-metadata.yml` / `Commit metadata`. The per-step `v3 server` / `v4 Postgres` labels stay, since they target specific backends. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Connor Tsui <connor@spiraldb.com>
1 parent 3f54d1f commit e0faa08

3 files changed

Lines changed: 38 additions & 15 deletions

File tree

‎.github/workflows/bench.yml‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -136,17 +136,24 @@ jobs:
136136
# `continue-on-error` so an OIDC / uv / connect hiccup never breaks the v3
137137
# pipeline. post-ingest.py mints the RDS IAM token internally (boto3) from
138138
# the assumed GitHubBenchmarkIngestRole; sslmode=verify-full validates the cert.
139+
#
140+
# ORDER MATTERS: "Install uv" runs BEFORE "Configure AWS credentials".
141+
# configure-aws-credentials persists the assumed ingest-role (rds-db:connect
142+
# only) as the job's ambient AWS creds; the uv setup compiles via sccache
143+
# (S3-backed), so running it after the role switch fails with S3 AccessDenied.
144+
# Installing uv first keeps sccache on the original S3-capable creds; the role
145+
# is assumed immediately before the ingest, which needs only rds-db:connect.
146+
- name: Install uv for v4 ingest
147+
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
148+
continue-on-error: true
149+
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
139150
- name: Configure AWS credentials for v4 ingest (OIDC)
140151
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
141152
continue-on-error: true
142153
uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
143154
with:
144155
role-to-assume: ${{ vars.GH_BENCH_INGEST_ROLE_ARN }}
145156
aws-region: ${{ vars.RDS_BENCH_REGION }}
146-
- name: Install uv for v4 ingest
147-
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
148-
continue-on-error: true
149-
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
150157
- name: Ingest results to v4 Postgres (best-effort)
151158
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
152159
continue-on-error: true

.github/workflows/v3-commit-metadata.yml renamed to .github/workflows/commit-metadata.yml

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
1-
# Posts a v3 ingest envelope with no records on every push to develop, so the
2-
# `commits` dim stays populated even when no benchmark ran.
1+
# Uploads commit metadata (an empty ingest envelope -- no benchmark records) on
2+
# every push to develop, to both the v3 server and the v4 Postgres, so the
3+
# `commits` dim stays populated even when no benchmark ran. Needed by both
4+
# backends, hence not named for either.
35

4-
name: v3 commit metadata
6+
name: Commit metadata
57

68
on:
79
push:
@@ -39,17 +41,24 @@ jobs:
3941
# a v4 failure never fails the job (promoted to required at cutover, PR-5.1).
4042
# Gated on the ingest-role ARN var (the assume-role input that MUST exist) so
4143
# it no-ops until v4 infra is wired.
44+
#
45+
# ORDER MATTERS: "Install uv" runs BEFORE "Configure AWS credentials".
46+
# configure-aws-credentials persists the assumed ingest-role (rds-db:connect
47+
# only) as the job's ambient AWS creds; the uv setup compiles via sccache
48+
# (S3-backed), so running it after the role switch fails with S3 AccessDenied.
49+
# Installing uv first keeps sccache on the original S3-capable creds; the role
50+
# is assumed immediately before the ingest, which needs only rds-db:connect.
51+
- name: Install uv for v4 ingest
52+
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
53+
continue-on-error: true
54+
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
4255
- name: Configure AWS credentials for v4 ingest (OIDC)
4356
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
4457
continue-on-error: true
4558
uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
4659
with:
4760
role-to-assume: ${{ vars.GH_BENCH_INGEST_ROLE_ARN }}
4861
aws-region: ${{ vars.RDS_BENCH_REGION }}
49-
- name: Install uv for v4 ingest
50-
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
51-
continue-on-error: true
52-
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
5362
- name: Ingest commit metadata to v4 Postgres (best-effort)
5463
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
5564
continue-on-error: true

‎.github/workflows/sql-benchmarks.yml‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -689,17 +689,24 @@ jobs:
689689
# input that MUST exist for OIDC to succeed; it no-ops until v4 infra is wired), and
690690
# every step is continue-on-error. post-ingest.py mints the RDS IAM token (boto3) from
691691
# the assumed GitHubBenchmarkIngestRole; sslmode=verify-full validates the cert.
692+
#
693+
# ORDER MATTERS: "Install uv" runs BEFORE "Configure AWS credentials".
694+
# configure-aws-credentials persists the assumed ingest-role (rds-db:connect
695+
# only) as the job's ambient AWS creds; the uv setup compiles via sccache
696+
# (S3-backed), so running it after the role switch fails with S3 AccessDenied.
697+
# Installing uv first keeps sccache on the original S3-capable creds; the role
698+
# is assumed immediately before the ingest, which needs only rds-db:connect.
699+
- name: Install uv for v4 ingest
700+
if: inputs.mode == 'develop' && vars.GH_BENCH_INGEST_ROLE_ARN != ''
701+
continue-on-error: true
702+
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
692703
- name: Configure AWS credentials for v4 ingest (OIDC)
693704
if: inputs.mode == 'develop' && vars.GH_BENCH_INGEST_ROLE_ARN != ''
694705
continue-on-error: true
695706
uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
696707
with:
697708
role-to-assume: ${{ vars.GH_BENCH_INGEST_ROLE_ARN }}
698709
aws-region: ${{ vars.RDS_BENCH_REGION }}
699-
- name: Install uv for v4 ingest
700-
if: inputs.mode == 'develop' && vars.GH_BENCH_INGEST_ROLE_ARN != ''
701-
continue-on-error: true
702-
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
703710
- name: Ingest results to v4 Postgres (best-effort)
704711
if: inputs.mode == 'develop' && vars.GH_BENCH_INGEST_ROLE_ARN != ''
705712
continue-on-error: true

0 commit comments

Comments
 (0)