Skip to content

Add release workflows#4583

Open
ss-es wants to merge 14 commits into
mainfrom
ss/add-release-workflows
Open

Add release workflows#4583
ss-es wants to merge 14 commits into
mainfrom
ss/add-release-workflows

Conversation

@ss-es

@ss-es ss-es commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Adds CI workflows for a release process.

In summary, we will now have an issue tracker for each release, with workflows that automatically track the divergence from main (so it is easier to see which commits have been backported or not). There is also some UX improvements for the tagging process, with a /tag (and in the same vein, /promote) command that can be used on the issue page.

Release branches are now release-X.Y.Z, where X.Y should correspond to the protocol version intended to run from this release. Tags now take the form of X.Y.PHASE.PATCH, where PATCH is automatically incremented via the /tag command.

The full process is described in RELEASES.md

@gemini-code-assist

Copy link
Copy Markdown
Contributor
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

@claude

claude Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Claude finished @ss-es's task in 3m 32s —— View job


Code Review: Add Release Workflows (PR #4583)

  • Gather context and read changed files
  • Review workflow files
  • Review release tracker script
  • Review documentation changes
  • Post detailed review feedback

Summary

This PR introduces a comprehensive release management system: issue-based tracker automation, /tag and /promote slash commands, Docker promotion with environment-gated approvals, backport candidate tracking, and migration from YYYYMMDD to MAJOR.MINOR.PHASE.PATCH versioning. The architecture is well thought out — release branches get lifecycle automation (create → track → tag → promote → teardown), and the documentation in RELEASES.md is thorough.

Findings

Bug (high):

  • list_commits uses --merges which produces empty results on this repo. This repo uses squash merges (git log --merges --first-parent returns nothing on main), so the "Backport candidates" and "Forward-port candidates" sections will always be empty. See inline comment.

Minor:

  • parse_skip_list (line 188) is dead code — only read_skips_full is called. Inline.
  • cmd_post_promote is implemented and wired up in argparse but never invoked by any workflow — promotion completion comments won't be posted to tracker issues. Inline.
  • tag-release.yml has no concurrency key — two simultaneous /tag comments could race (safe but confusing error). Inline.
  • render_candidates skip matching uses bidirectional prefix comparison that could be simplified. Inline.

What looks good

  • Security: Authorization gating (OWNER|MEMBER|COLLABORATOR) on all slash commands. pull_request_target usage is safe (only types: [closed], no PR head checkout). All user-controlled inputs are passed via env: mapping, not direct ${{ }} interpolation in run: blocks.
  • Idempotency: Label creation, tracker issue creation, and branch deletion handling are all idempotent — safe to retry.
  • Promotion progression enforcement: check-progression job in promote-docker-tag.yml validates that tags are promoted through stages in order (decaf.rc → decaf → mainnet.rc → mainnet) with a skip-progression escape hatch for hotfixes.
  • Documentation: RELEASES.md is comprehensive and well-organized. The doc/software-releases.md redirect keeps the old path discoverable.
  • Binary upgrade test migration: Clean transition from YYYYMMDD to X.Y.Z.N tag format with proper version-aware sorting (tuple comparison instead of lexicographic).

@ss-es ss-es marked this pull request as draft June 24, 2026 10:23
Comment thread scripts/release-tracker.py Outdated
Comment thread scripts/release-tracker.py Outdated
Comment thread scripts/release-tracker.py Outdated
Comment thread .github/workflows/tag-release.yml
Comment thread scripts/release-tracker.py Outdated
@ss-es ss-es marked this pull request as ready for review June 26, 2026 06:19
@claude

claude Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Claude finished @ss-es's task in 6m 13s —— View job


Code Review: Add Release Workflows (PR #4583)

  • Read existing review comments and gather context
  • Read CLAUDE.md for repo-specific guidelines
  • Review all changed files in detail
  • Check for issues not already flagged by previous review
  • Resolve own threads where issues are fixed
  • Post final review feedback

Summary

Since the last review, scripts/release-tracker.py was removed and its functionality was inlined into update-release-tracker.yml as a bash script. The previous review's findings about the Python file (--merges bug, parse_skip_list dead code, cmd_post_promote unused, bidirectional prefix match) are all resolved by this refactoring. The inline bash version uses --no-merges which is correct for this repo's squash-merge workflow.

I resolved the open concurrency thread on tag-release.yml — the author is right that it's harmless (the second push just fails).

New Findings

Bug:

  • RELEASES.md manual fallback uses wrong workflow input names. Line 154 documents -f stage=decaf.canary -f tag=0.4.0.5 but promote-docker-tag.yml defines its inputs as floating-tag and release-tag. Anyone following the docs would get an error. See inline comment.

Security (medium):

  • /promote stage value is not validated before dispatching. The stage extracted from the issue comment (awk '{print $2}') is passed directly to gh workflow run promote-docker-tag.yml -f floating-tag="$stage". Since GitHub's type: choice is only enforced in the UI, any string is accepted via API/CLI. A comment like /promote foobar would (1) use an unconfigured environment with no approval gate, (2) skip the check-progression case (no match = no check), and (3) tag Docker images as foobar. Requires OWNER/MEMBER/COLLABORATOR, so limited blast radius, but the progression/approval gates can be bypassed inadvertently. See inline comment.

Minor:

  • Backport error reporting regression. The previous backport.yml had a fail() function that commented on PRs when Claude's conflict resolution produced issues (remaining conflict markers, cargo check failure, etc.). The new version silently fails the Actions job — the PR author has to navigate to the workflow run to see what went wrong. Consider re-adding a simple "conflict resolution failed, see workflow run" comment at minimum.

What looks good

  • Refactoring from Python to inline bash: The release-tracker.py → inline bash migration simplifies the repo — no Python dependency, no separate script to maintain. The bash checklist() function is well-structured with proper cursor tracking and tick preservation.
  • tick-tracker job: Clean integration with backport.yml — the branch naming convention parsing (backport-<PR>-to-<target>) reliably identifies the original PR and determines backport vs forward-port direction.
  • Binary upgrade test migration: The move from YYYYMMDD to X.Y.Z.N tagging with proper version-aware sorting (tuple(int(p) for p in t.split("."))) is correct and handles all edge cases (HEAD on tag, oldest tag).
  • Idempotent label creation: All gh label create calls use 2>/dev/null || true — safe to retry.
  • build.yml tag pattern: [0-9]+.[0-9]+.[0-9]+.[0-9]+ correctly uses GitHub Actions' + quantifier (one or more of preceding character class).
  • Security model is solid — all slash commands gated on author association, user inputs passed via env: mapping (not ${{ }} interpolation in run: blocks), pull_request_target usage is safe (only on types: [closed]).

Comment thread RELEASES.md Outdated
Comment thread .github/workflows/update-release-tracker.yml Outdated
Comment thread RELEASES.md Outdated
tracker issue. The tracker issue is where you'll do everything that
follows.

2. **Run an initial tag.** Comment `/tag` on the tracker issue. This creates

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 2. can be fully automated after 1.? So can we just make this part of a workflow dispatch run or have at least a CLI convenience command? I think it's easier if everything happens in one place instead of having to run CLI commands and perform comments on github.


refresh() {
local issue="$1"
local body; body=$(gh issue view "$issue" --json body -q .body)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is difficult to review, or even just figure out what i does exactly and there's currently no way to run anything locally, or any tests because it's inlined into the workflow. I think it could be extracted into a python script for better readability and testability.

I think there may be some concurrency issues and the workflow has many triggers.

@@ -0,0 +1,43 @@
---

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently the issue is used as a kind of database and it's just a string. There's no real schema, versioning, migration etc. so I'm afraid if we need to change or fix anything it may get very messy. I think there must be a better way to do this.

I haven't thought about it much but for example the state could be kept as json (or similar) in the issue body (commented) and managed by a python script.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be resolved now -- the issue page itself no longer maintains any state

jobs:
on-create:
if: |
github.event_name == 'create' &&

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding workflow dispatch for this instead of pushing a branch and then name matching on the branch? The workflow could take any rev / branch / tag as input, plus the version and then create the branch on gihub


YYYYMMDD_TAG_PATTERN = "20[0-9][0-9][0-1][0-9][0-3][0-9]"
RELEASE_TAG_GLOB = "[0-9]*.[0-9]*.[0-9]*.[0-9]*"
RELEASE_TAG_RE = re.compile(r"^\d+\.\d+\.\d+\.\d+$")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fallback to YYYYMMDD if it doesn't find new tags?

Not part of this PR but we should rewrite it a bit anyway to pick up the versions that are deployed instead of tagged on github. Currently it moves the base version up too early.

@sveitser sveitser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the overall ideas are great and a significant improvement. I like the sticky github issue and interaction via slash commands.

I left some comments with suggestions and concerns.

I think my biggest concern is that this adds a lot of bash-in-yaml CI code that is almost untestable so I would prefer not to do that. Whenever a workflow runs it can use all the code in the repo so I see no reason not to script the complicated parts outside github yaml files. This also allows to add some tests for it to document important behaviors that are currently implicit and difficult to determine from looking at the code. We could for example add a python script that inputs and outputs the issue body (potentially containing a more structured artifact, as suggested).

Comment thread RELEASES.md Outdated
The tracker issue is the durable, branch-scoped surface for everything
release-related. It is the place where you run release commands, see status,
and discuss the release with the team. Its body is maintained automatically;
the bot edits sections between `<!-- BEGIN ... -->` / `<!-- END ... -->`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are no such markers now?

run: |
set -euo pipefail
git tag -a "$TAG" -m "Release $TAG"
git push origin "refs/tags/$TAG"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this workflow is missing the step to create a github release.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to create a github release on every tag, since we likely will use tags for triggering builds for devnet as well now

@sveitser

Copy link
Copy Markdown
Collaborator

Since almost nothing on the workflow is specific to this repo why don't we first put it into a separate repo build some trivial docker images then we can test the whole workflow very quickly, including promotion, docker tag protection etc. and avoid polluting this repo and waiting for long builds.

@ss-es

ss-es commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@claude I've rearchitectured some of this, can you take another look?

@claude

claude Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Claude encountered an error after 59s —— View job


Code Review: Add Release Workflows (PR #4583) — Re-review

  • Gather context and read the current diff
  • Review workflow files for architectural changes
  • Review documentation and scripts
  • Check previous review findings are resolved
  • Post detailed review feedback

Reading the current state of the PR...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants