Skip to content

ci(e2e): refresh e2e timings manifest#10462

Merged
davidfirst merged 1 commit into
masterfrom
update-e2e-timings-2026-07-01
Jul 1, 2026
Merged

ci(e2e): refresh e2e timings manifest#10462
davidfirst merged 1 commit into
masterfrom
update-e2e-timings-2026-07-01

Conversation

@GiladShoham

Copy link
Copy Markdown
Member

Automated biweekly refresh of scripts/e2e-test-timings.json from the last 12 e2e runs (5 entries drifted meaningfully). Generated by the e2e_timings_refresh scheduled workflow.

Copilot AI review requested due to automatic review settings July 1, 2026 04:02
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

ci(e2e): refresh e2e timings manifest

⚙️ Configuration changes 🕐 Less than 10 minutes

Grey Divider

AI Description

• Refresh per-spec e2e runtime estimates from the latest 12 CI runs.
• Update the timings manifest used for smarter e2e job sharding.
• Keep CI e2e scheduling aligned with recent runtime drift.
Diagram

graph TD
  A["Scheduled workflow"] --> B["e2e timings refresh job"] --> C["scripts/e2e-test-timings.json"] --> D["CI e2e runner"] --> E["E2E spec shards"]
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Compute timings dynamically per CI run (no checked-in manifest)
  • ➕ Always reflects current runtime without PR churn
  • ➕ Avoids large diffs of generated numbers
  • ➖ Less reproducible: shard planning can vary run-to-run
  • ➖ Requires extra CI steps and a source of truth for historical runtimes
2. Store timings as a versioned artifact (e.g., in CI cache/object storage)
  • ➕ Reduces git noise while keeping historical versions
  • ➕ Can support rollbacks to a previous baseline
  • ➖ Adds operational complexity and permissions/retention concerns
  • ➖ Consumers must handle download failures and fallbacks

Recommendation: Keeping a checked-in manifest is a reasonable tradeoff for deterministic sharding and easy inspection; the scheduled refresh keeps it current. If PR noise becomes a recurring pain point, consider artifact-based storage as a next step.

Files changed (1) +180 / -175

Other (1) +180 / -175
e2e-test-timings.jsonUpdate e2e per-spec duration estimates +180/-175

Update e2e per-spec duration estimates

• Refreshes the mapping of e2e spec paths to expected runtimes based on recent CI executions. Values drift across many specs to keep CI sharding and scheduling aligned with current performance.

scripts/e2e-test-timings.json

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the scripts/e2e-test-timings.json manifest used to balance CircleCI E2E test workloads across parallel nodes, reflecting refreshed per-file wall-clock estimates derived from the last 12 successful E2E runs.

Changes:

  • Updated many per-test-file duration estimates in scripts/e2e-test-timings.json based on recent CI observations.
  • Included new/previously-untracked E2E files in the manifest (while keeping the manifest valid JSON and preserving the 15s minimum floor behavior expected by the generator script).

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Qodo Logo

@davidfirst davidfirst merged commit c934235 into master Jul 1, 2026
14 checks passed
@davidfirst davidfirst deleted the update-e2e-timings-2026-07-01 branch July 1, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants