Skip to content

Commit f47bf54

Browse files
a10yclaude
andauthored
ci: cut wasted minutes from measurement_id and WASM jobs (#8776)
## Summary Three targeted fixes from profiling recent CI runs (job/step timings + logs from PR runs of `Linters and Tests`): - **`measurement_id golden vectors`: 6.2 min → ~20 s.** The `setup-uv` action defaults to `uv sync --all-extras --dev`, which builds the vortex-data PyO3 extension via maturin on an uncached `ubuntu-latest` runner — ~6 min of the job (verified in job logs: `Run uv sync --all-extras --dev` → `Building vortex-data`). The test itself takes ~1 s and uses `uv run --no-project`, so it needs no workspace packages. Pass `sync: false`, matching the existing `python-cuda-test` job. - **WASM integration smoke test: ~5.2 min → expected ~1–2 min.** It was the only Rust-compiling job left on plain `ubuntu-latest` with no compiler cache; ~4.5 min is a cold `cargo build`. Moved to the prebuilt runs-on image with S3 sccache, following the same pattern as the other build jobs (fork fallback to `ubuntu-latest` preserved; `rustup target add wasm32-wasip1` mirrors what the wasm32 build job does). - **Windows sccache observability.** `Rust tests (windows-x64)` is the ci.yml PR critical path (consistently 6.8–7.7 min across the last 8 PR runs, ~5 min of it compile). The sccache server demonstrably starts, but logs give no way to see hit rates. Added an `sccache --show-stats` step (`if: always()`, main repo only) so cache effectiveness on Windows is visible per run. ## Not included (infra-side observations from the same investigation) - Codspeed PR runs take 9–14 min but shards compute for only 0.8–4 min — they queue 5.5–9.8 min for `amd64-medium` runners while concurrent ci.yml jobs get runners in <1 min. One PR push requests ~26 runs-on VMs at once; this looks like a concurrent-instance/pool cap in `.github-private` (this repo's `runs-on.yml` only defines images). Raising capacity there, or consolidating the 8 Codspeed shards to 4, would cut the largest remaining chunk of PR wall clock. - The Windows pool uses `m8i-flex.2xlarge` (8 vCPU); a size bump would directly shrink the PR critical path since compile dominates and linking 53 test binaries is uncacheable. ## Checks - `yamllint --strict -c .yamllint.yaml .github/workflows/ci.yml` — passes. - Workflow-only change; no Rust/Python code touched, so no cargo/pytest checks were run per repo guidance. The WASM job change is best validated by this PR's own CI run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Andrew Duffy <andrew@a10y.dev> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 2cade61 commit f47bf54

1 file changed

Lines changed: 20 additions & 3 deletions

File tree

.github/workflows/ci.yml

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,11 @@ jobs:
146146
timeout-minutes: 10
147147
steps:
148148
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
149+
# sync: false — the test runs via `uv run --no-project` and needs no workspace
150+
# packages; the default `uv sync` builds the vortex-data Rust extension (~6 min).
149151
- uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6
152+
with:
153+
sync: false
150154
- name: Pytest - measurement_id golden vectors
151155
run: |
152156
uv run --no-project --with pytest --with xxhash \
@@ -415,6 +419,10 @@ jobs:
415419
--exclude compress-bench --exclude xtask --exclude vortex-datafusion `
416420
--exclude gpu-scan-cli --exclude vortex-sqllogictest
417421
422+
- name: sccache stats
423+
if: always() && github.repository == 'vortex-data/vortex'
424+
run: sccache --show-stats
425+
418426
- name: Alert incident.io
419427
if: failure() && github.event_name == 'push' && github.ref == 'refs/heads/develop'
420428
uses: ./.github/actions/alert-incident-io
@@ -584,14 +592,23 @@ jobs:
584592

585593
wasm-integration:
586594
name: "WASM integration smoke test"
587-
runs-on: ubuntu-latest
588595
timeout-minutes: 30
596+
runs-on: >-
597+
${{ github.repository == 'vortex-data/vortex'
598+
&& format('runs-on={0}/runner=amd64-medium/image=ubuntu24-full-x64-pre-v2/extras=s3-cache/tag=wasm-integration', github.run_id)
599+
|| 'ubuntu-latest' }}
589600
steps:
601+
- uses: runs-on/action@v2
602+
if: github.repository == 'vortex-data/vortex'
603+
with:
604+
sccache: s3
590605
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
591-
- uses: ./.github/actions/setup-rust
606+
- uses: ./.github/actions/setup-prebuild
592607
with:
593-
repo-token: ${{ secrets.GITHUB_TOKEN }}
608+
enable-sccache: "true"
594609
targets: "wasm32-wasip1"
610+
- name: Install wasm32-wasip1 target
611+
run: rustup target add wasm32-wasip1
595612
- name: Setup Wasmer
596613
shell: bash
597614
run: |

0 commit comments

Comments
 (0)