Skip to content

Add duplicate run detection to skip redundant builds#253

Open
Biont wants to merge 6 commits into
mainfrom
feature/build-n-dist-duplicate-run-detection
Open

Add duplicate run detection to skip redundant builds#253
Biont wants to merge 6 commits into
mainfrom
feature/build-n-dist-duplicate-run-detection

Conversation

@Biont

@Biont Biont commented Jun 3, 2026

Copy link
Copy Markdown
Member

Skip redundant builds in build-and-distribute when SHA is already compiled

Skip the entire compilation pipeline when the build branch already contains a build for the current commit SHA, saving several minutes of Node/PHP setup on duplicate workflow runs (re-runs, race conditions between concurrent triggers, or any other scenario where the same SHA is built more than once).

Rationale

The intent of build&distribute was always to maintain the build branch as the stable source of truth. It gives us a traceable, clean artifact that is not buried in a million GHA executions and it does not expire after 90 days.

What we see in practice though is that the coupling between dev/foo(<- the source branch) and foo (<- the build branch) is a surprisingly annoying obstacle to overcome when building reliable CI pipelines.
So there has always been an incentive to keep workflow artifacts around as a side-channel to the build-branch-commit (also because it offers a stable plugin/theme subfolder which we do not have on feature branches).

  • On push to dev/foo, we cannot call e2e tests that require a built package since we push to the dev branch
  • We cannot call e2e tests on foo because that has no direct connection to the dev branch: It will execute, but failures will not show up on the PR, so they'd be an costly, easy-to-miss optional signal that does not actually help QA efforts
  • We do not want every build consumer to produce their own build&distribute run because this is wasteful and redundant.
  • However, we cannot properly decouple build & distribute from e2e tests and still chain them so that e2e only runs when we know that a build has completed. While it is possible to use a (completed) workflow_run trigger, the handover of the build branch checkout is still not trivial

A clean solution would be that indeed every consumer runs build&distribute and then uses the artifact it produces. But as I said this is redundant and wasteful.
This PR attempts to make it less wasteful and thereby enable this as a clean pattern.

Ideally, it would turn a ~5min execution into a <1min execution that does not add any noise to the actual build branch, so it would be safe to call it repeatedly.

Please check if the PR fulfills these requirements

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes/features)
  • Docs have been added/updated (for bug fixes/features)

What kind of change does this PR introduce?

Feature / optimization

What is the current behavior?

Every workflow run performs a full build regardless of whether the build branch already contains compiled assets for the triggering commit. Two runs for the same SHA produce identical output but consume identical runner time.

What is the new behavior?

Duplicate run detection

A new "Check if already built for this SHA" step runs immediately before the build branch is checked out. It reads the remote build branch without checking it out and looks for the current commit SHA in:

  • SHA: header in style.css — WordPress themes
  • SHA: header in the main plugin PHP file — WordPress plugins

If the SHA is found, SKIP_BUILD=true is set and all compilation steps are skipped. The workflow then checks out the existing build branch (without wiping it) and proceeds directly to artifact packaging and upload, so the artifact output is always populated for downstream jobs.

Library projects (no theme or plugin header) are unaffected and always rebuild.

SHA upsert

The SHA: field is now written into plugin/theme headers even when it is absent from the source file. Previously the sed substitution was a no-op if the field did not exist, which would prevent the skip gate from ever firing. The new behaviour inserts SHA: <hash> after the Version: line on the first build, enabling duplicate detection for all subsequent runs.

Earlier preparation steps

"Determine package type" and "Prepare WordPress Plugin/Theme" have been moved to immediately after the source checkout, before any PHP or Node toolchain setup. This means a skip-gate hit now short-circuits the run before the most expensive steps (PHP setup, Composer install, Node setup, npm ci, npm run build) are even reached.

Does this PR introduce a breaking change?

No. The artifact output and build branch push behaviour are unchanged for normal (non-duplicate) runs.

Changes

  • .github/workflows/build-and-distribute.yml:
    • Added "Check if already built for this SHA" step (after branch validation, before checkout)
    • Modified "Checkout the build branch with clean slate" to use a fast path when SKIP_BUILD=true: checks out the existing build branch without wiping it
    • Moved "Determine package type", "Prepare WordPress Plugin", "Prepare WordPress Theme" to immediately after checkout (before toolchain setup); all three now have if: SKIP_BUILD != 'true'
    • Updated SHA sed in both Prepare steps to upsert: replaces existing SHA: line or inserts after Version: if absent
    • Added if: ${{ env.SKIP_BUILD != 'true' }} to every build/compile step from "Check for composer.json" through "Git add, commit, and push"; artifact steps are left unconditional
  • docs/build-and-distribute.md:
    • Updated intro step list to reflect the new skip check (step 2) and SHA upsert
    • Added "Duplicate run detection" subsection under "Build process details" explaining the gate, its scope, and the library limitation
    • Clarified SHA: behaviour in "Version Management" subsection

Introduce a `Check if already built for this SHA` step that
inspects the remote build branch before any toolchain setup.
If the current commit SHA is already embedded in `style.css`
(themes) or the main plugin PHP file (plugins), the entire
compilation pipeline is skipped and the workflow proceeds
directly to packaging the existing build branch content.

Key changes:
- New early-exit step reads `SHA:` header from the remote
  build branch and sets `SKIP_BUILD=true` when a match is
  found, bypassing install, compile, and commit steps
- All subsequent build steps are guarded with
  `SKIP_BUILD != 'true'` conditions
- `Determine package type` and `Prepare WordPress *` steps
  moved earlier (before npm/composer install) so the SHA
  can be embedded before compilation
- SHA field handling changed from a plain replace to an
  upsert: inserts a new `SHA:` line after `Version:` when
  the field is absent, replaces it when present
- Downstream jobs remain unaffected; the artifact output is
  always populated regardless of whether the build was fresh
  or reused
- Library projects do not embed a `SHA:` field and always
  rebuild
- Trailing whitespace cleaned up throughout the file
@Biont Biont requested a review from a team June 3, 2026 10:40

@tyrann0us tyrann0us left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new "Check if already built for this SHA" step runs immediately before the build branch is checked out. It reads the remote build branch without checking it out (fetch-depth: 0 makes all remote objects available) and looks for the current commit SHA in […]:

I was about to ask you to update the PR description, since the current approach no longer uses fetch-depth, but then I checked your original commit bbdeb88 (this PR), and it doesn't use it either. 😆
But in any case, the description is outdated anyway because my changes bbdeb88..9ea6215 (this PR) moved and merged steps.

I'm approving, but it would be great if you could update the description to avoid confusion later. Thanks! 🙏🏽

Comment thread .github/workflows/build-and-distribute.yml Outdated
Biont and others added 3 commits June 4, 2026 08:59
Co-authored-by: Philipp Bammes <8144115+tyrann0us@users.noreply.github.com>
Signed-off-by: Moritz Meißelbach <arbelzapf@gmail.com>
Prevent duplicate builds when the same SHA is triggered more than
once (e.g., push + manual dispatch racing). By setting
`cancel-in-progress: false`, a queued second run will detect the
already-built artifact and resolve without repeating the full build.

- Add `concurrency` group scoped to SHA + branch name on the job
- Update docs to reflect the new serialization behavior and recommend
  `cancel-in-progress: false` in caller workflows
…un-detection' into feature/build-n-dist-duplicate-run-detection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants