Skip to content

Skip missing upstream branches in scheduled build#382

Merged
qu0b merged 2 commits into
masterfrom
qu0b/skip-missing-upstream-branches
May 18, 2026
Merged

Skip missing upstream branches in scheduled build#382
qu0b merged 2 commits into
masterfrom
qu0b/skip-missing-upstream-branches

Conversation

@qu0b
Copy link
Copy Markdown
Member

@qu0b qu0b commented May 18, 2026

Summary

The hourly Scheduled workflow has been failing on every run because branches.yaml lists branches that don't (yet) exist on the upstream repo — most recently geth/bal-devnet-7. Each failure also pings Discord, so the signal-to-noise on real breakages is poor.

Root cause

process_commits in .github/workflows/scheduled.yml already detects a missing upstream ref via the GitHub API and logs:

[LINE:28] Error fetching commit hash for ethereum/go-ethereum#bal-devnet-7, skipping.

…but the downstream loops still iterate over every config in config.yaml. For the missing entry:

  1. COMMIT_HASH comes back empty.
  2. The Docker Hub check queries the malformed image ethpandaops/geth:bal-devnet-7- (trailing dash).
  3. That tag (unsurprisingly) doesn't exist, so the entry is classified as "needs build" and added to the deploy matrix.
  4. The deploy job fails at actions/checkout with A branch or tag with the name 'bal-devnet-7' could not be found.

Confirmed against the latest run (gh run 26026938948).

Fix

Two narrow changes inside the check job:

  • Docker Hub existence loopcontinue when COMMIT_HASH is empty/null. Avoids wasted Docker Hub queries against malformed tags.
  • Build-config generation loop — same skip, plus a ::warning:: annotation and a row appended to $GITHUB_STEP_SUMMARY (under a ## Skipped builds (upstream ref missing) heading written once).

The workflow stays green, the Discord notification step (if: cancelled() || failure()) no longer fires for missing-branch noise, and every skipped branch is plainly visible on the run page.

Self-healing

No state is persisted. The moment the branch is pushed upstream, the GitHub API returns a SHA on the next hourly cron → COMMIT_HASH is non-empty → the skip branch isn't taken → Docker Hub reports the proper <tag>-<sha> image missing → the build runs and pushes normally. No manual intervention needed when the branch reappears.

Test plan

  • Trigger gh workflow run scheduled.yml after merge.
  • Confirm the run is green even though geth/bal-devnet-7 is still missing upstream.
  • Confirm a ::warning:: annotation appears on the run page for the missing entry.
  • Confirm the run summary contains a Skipped builds (upstream ref missing) table listing the geth row (and any others).
  • Confirm an existing branch (e.g. geth/master) still builds normally if its image is missing on Docker Hub.
  • Confirm no Discord notification fires for the skipped entries.

When a branch listed in branches.yaml doesn't exist on the upstream
repository (e.g. geth/bal-devnet-7 before it lands), process_commits
already detects it and skips writing a commit hash. The Docker Hub
existence check and config-generation loop still iterated over those
entries though, ended up querying for a malformed "<tag>-" image, found
it "missing", and queued a build that immediately failed at
actions/checkout. Every hourly cron then produced a red run and a
Discord ping despite there being nothing actionable.

Skip empty-hash configs in both loops. For the build-generation loop,
emit a ::warning:: and a job-summary table so the missing branches are
visible on the run page without failing the workflow. Self-healing: the
moment the branch lands, the API returns a SHA and the next run builds
it normally.
@qu0b-reviewer
Copy link
Copy Markdown

qu0b-reviewer Bot commented May 18, 2026

🤖 qu0b-reviewer

The PR is clean — the two guards (Docker Hub check loop and config-generation loop) correctly skip configs whose upstream ref process_commits failed to resolve, with graceful/no-op fallback paths throughout. The SKIPPED_HEADER_WRITTEN sentinel for the summary table is correct (no double-header, no output on zero skips).

Looks good. 🦋


Reviewed @ d9bda370
"All non-trivial abstractions, to some degree, are leaky." — Joel Spolsky

@qu0b qu0b merged commit e0b0476 into master May 18, 2026
3 checks passed
@qu0b qu0b deleted the qu0b/skip-missing-upstream-branches branch May 18, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants