EPIC-07: Add CI perf regression and repro replay gates#59
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d68383e45
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "arch": "x64", | ||
| "osType": "Linux", | ||
| "osRelease": "6.6.87.2-microsoft-standard-WSL2", | ||
| "cpuModel": "AMD Ryzen 7 9800X3D 8-Core Processor", |
There was a problem hiding this comment.
Regenerate perf baseline on CI runner environment
This baseline artifact was captured on a local WSL host (cpuModel is a desktop Ryzen and the file also records Node v20), but the perf gate in .github/workflows/ci.yml runs on ubuntu-latest with Node 22, so the comparator is gating absolute timing deltas against a different machine/runtime than the one used in CI. Because bench-ci-compare enforces hard thresholds on timing.mean, this can create persistent false regressions (or hide real ones) on every PR; the committed baseline should be generated in the same CI environment as the gate.
Useful? React with 👍 / 👎.
Summary
bench:ci)bench:ci:compare)benchmarks/ci-baseline/Validation
npm run lintnpm run typechecknpm run buildnpm testnpm run test:e2enpm run test:e2e:reducednpm run test:native:smokenpm run bench:ci -- --output-dir /tmp/rezi-bench-ci-prnpm run bench:ci:compare -- --baseline benchmarks/ci-baseline/results.json --current /tmp/rezi-bench-ci-pr/results.json --config benchmarks/ci-baseline/config.json --report-json /tmp/rezi-bench-ci-pr/compare.json --report-md /tmp/rezi-bench-ci-pr/compare.md