Skip to content

Commit 8dfe2ca

Browse files
dlvenabledlvenable
committed
Updates the release process to support a dedicated fork for creating PRs per recent changes to the OpenSearch project org rules.
The change is both to the release and release-prepare-branch GitHub Actions. The new approach uses a dedicated fork for creating PRs. Access to the fork is granted via a personal access token. The release workflow now longer uses the app token for stronger security. This also updates related documentation for releases. Resolves opensearch-project#6912. Signed-off-by: David Venable <dlv@amazon.com>
1 parent 4d2bc56 commit 8dfe2ca

3 files changed

Lines changed: 212 additions & 31 deletions

File tree

‎.github/workflows/release-prepare-branch.yml‎

Lines changed: 53 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,33 @@
1010
# This workflow automates the preparation of patch release branches.
1111
# It validates the release branch, updates gradle.properties to the next version
1212
# generates the THIRD-PARTY file, and creates a pull request with all changes.
13+
#
14+
# The OpenSearch project does not permit this repository's workflows to push
15+
# branches and open pull requests against itself. Instead, the pull request
16+
# branch is pushed to a fork and the PR is opened from that fork against this
17+
# repository. This requires two settings to be configured:
18+
#
19+
# vars.RELEASE_FORK_REPOSITORY - the fork the PR branch is pushed to, as
20+
# 'owner/repo'. It must be a real GitHub fork
21+
# of this repository (same fork network), as
22+
# cross-repository PRs are only allowed within
23+
# a fork network.
24+
# secrets.RELEASE_FORK_TOKEN - a classic personal access token with the
25+
# 'public_repo' (or 'repo' for private
26+
# repositories) and 'workflow' scopes. The
27+
# action uses it to push the branch to the
28+
# fork and to open the PR against this
29+
# repository, so its owner must have write
30+
# access to the fork. The 'workflow' scope is
31+
# required because the pushed branch may modify
32+
# files under .github/workflows; GitHub rejects
33+
# such pushes from a token without it. A
34+
# fine-grained token does not work here because
35+
# the single token must act on two repositories.
36+
#
37+
# The PR base is always this repository (the one running the workflow). To test
38+
# on your own fork, run this workflow from your fork and point
39+
# RELEASE_FORK_REPOSITORY at a fork of your fork within the same network.
1340

1441
name: Prepare Release Branch
1542

@@ -27,6 +54,12 @@ jobs:
2754
steps:
2855
- name: Checkout Data Prepper
2956
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
57+
with:
58+
# Do not persist the checkout credentials in the local git config.
59+
# When pushing to a fork, peter-evans/create-pull-request adds its own
60+
# authentication, and a persisted Authorization header would collide
61+
# with it, causing 'Duplicate header: "Authorization"' (HTTP 400).
62+
persist-credentials: false
3063

3164
- name: Validate release branch
3265
id: validate_branch
@@ -50,6 +83,20 @@ jobs:
5083
echo "✅ Valid release branch: $BRANCH_NAME"
5184
echo "branch=$BRANCH_NAME" >> $GITHUB_OUTPUT
5285
86+
- name: Validate fork configuration
87+
run: |
88+
if [ -z "${{ vars.RELEASE_FORK_REPOSITORY }}" ]; then
89+
echo "::error::The RELEASE_FORK_REPOSITORY variable is not set."
90+
echo "Set it to the 'owner/repo' of a fork of this repository that the release PR branch will be pushed to."
91+
exit 1
92+
fi
93+
if [ -z "${{ secrets.RELEASE_FORK_TOKEN }}" ]; then
94+
echo "::error::The RELEASE_FORK_TOKEN secret is not set."
95+
echo "Set it to a token with write access to ${{ vars.RELEASE_FORK_REPOSITORY }} and permission to create pull requests against this repository."
96+
exit 1
97+
fi
98+
echo "Pull request branch will be pushed to fork: ${{ vars.RELEASE_FORK_REPOSITORY }}"
99+
53100
- name: Set up JDK
54101
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
55102
with:
@@ -91,18 +138,12 @@ jobs:
91138
92139
echo "Successfully generated THIRD-PARTY file"
93140
94-
- name: GitHub App token
95-
id: github_app_token
96-
uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2
97-
with:
98-
app_id: ${{ secrets.APP_ID }}
99-
private_key: ${{ secrets.APP_PRIVATE_KEY }}
100-
101141
- name: Create Pull Request
102142
id: create_pr
103143
uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6
104144
with:
105-
token: ${{ steps.github_app_token.outputs.token }}
145+
token: ${{ secrets.RELEASE_FORK_TOKEN }}
146+
push-to-fork: ${{ vars.RELEASE_FORK_REPOSITORY }}
106147
add-paths: |
107148
gradle.properties
108149
THIRD-PARTY
@@ -129,6 +170,7 @@ jobs:
129170
130171
After merging this PR:
131172
- [ ] Prepare release notes (see [release notes script](release/script/release-notes/README.md))
173+
- [ ] Open an issue in the [opensearch-project/.github](https://github.com/opensearch-project/.github) repository requesting the release tag `${{ steps.version.outputs.version }}` be created and pushed.
132174
- [ ] Run the [Release Artifacts workflow](https://github.com/opensearch-project/data-prepper/actions/workflows/release.yml)
133175
- [ ] Approve the release issue (requires 2 maintainer approvals)
134176
- [ ] Update the draft release with release notes
@@ -153,8 +195,9 @@ jobs:
153195
run: |
154196
echo "::error::Failed to create pull request"
155197
echo "This could be due to:"
156-
echo " - Missing or invalid GitHub App credentials"
157-
echo " - Insufficient permissions"
198+
echo " - An invalid or expired RELEASE_FORK_TOKEN"
199+
echo " - The token lacking write access to ${{ vars.RELEASE_FORK_REPOSITORY }} or permission to create pull requests"
200+
echo " - RELEASE_FORK_REPOSITORY (${{ vars.RELEASE_FORK_REPOSITORY }}) not being a fork of this repository in the same fork network"
158201
echo " - Network issues"
159202
echo "Please check the logs above for more details"
160203
exit 1

‎.github/workflows/release.yml‎

Lines changed: 80 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,32 @@
77
# compatible open source license.
88
#
99

10+
# This workflow builds and validates the release artifacts, drafts the GitHub
11+
# release, and opens the changelog pull request.
12+
#
13+
# The OpenSearch project does not permit this repository's workflows to push
14+
# branches and open pull requests against itself. Instead, the changelog PR
15+
# branch is pushed to a fork and the PR is opened from that fork against this
16+
# repository. This requires two settings to be configured:
17+
#
18+
# vars.RELEASE_FORK_REPOSITORY - the fork the PR branch is pushed to, as
19+
# 'owner/repo'. It must be a real GitHub fork
20+
# of this repository (same fork network), as
21+
# cross-repository PRs are only allowed within
22+
# a fork network.
23+
# secrets.RELEASE_FORK_TOKEN - a classic personal access token with the
24+
# 'public_repo' (or 'repo' for private
25+
# repositories) and 'workflow' scopes. The
26+
# action uses it to push the branch to the
27+
# fork and to open the PR against this
28+
# repository, so its owner must have write
29+
# access to the fork. The 'workflow' scope is
30+
# required because the pushed branch may modify
31+
# files under .github/workflows; GitHub rejects
32+
# such pushes from a token without it. A
33+
# fine-grained token does not work here because
34+
# the single token must act on two repositories.
35+
1036
name: Release Artifacts
1137

1238
on:
@@ -30,9 +56,60 @@ permissions:
3056
pull-requests: write
3157

3258
jobs:
59+
# Verifies the release preconditions
60+
# 1. The release tag must already exist. A maintainer requests the tag via an issue in the
61+
# opensearch-project/.github repository before running this workflow.
62+
# 2. The fork settings used to open the changelog pull request must be set.
63+
verify-preconditions:
64+
runs-on: ubuntu-latest
65+
timeout-minutes: 5
66+
67+
steps:
68+
- name: Checkout Data Prepper
69+
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
70+
- name: Get Version
71+
run: grep '^version=' gradle.properties >> $GITHUB_ENV
72+
- name: Verify release tag exists
73+
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
74+
with:
75+
github-token: ${{ github.TOKEN }}
76+
script: |
77+
const tag = '${{ env.version }}';
78+
try {
79+
await github.rest.git.getRef({
80+
owner: context.repo.owner,
81+
repo: context.repo.repo,
82+
ref: `tags/${tag}`
83+
});
84+
core.info(`Found tag ${tag}.`);
85+
} catch (error) {
86+
if (error.status === 404) {
87+
core.setFailed(
88+
`Release tag '${tag}' does not exist. Request the tag by opening an issue in the ` +
89+
`opensearch-project/.github repository before running this workflow.`
90+
);
91+
} else {
92+
throw error;
93+
}
94+
}
95+
- name: Validate fork configuration
96+
run: |
97+
if [ -z "${{ vars.RELEASE_FORK_REPOSITORY }}" ]; then
98+
echo "::error::The RELEASE_FORK_REPOSITORY variable is not set."
99+
echo "Set it to the 'owner/repo' of a fork of this repository that the changelog PR branch will be pushed to."
100+
exit 1
101+
fi
102+
if [ -z "${{ secrets.RELEASE_FORK_TOKEN }}" ]; then
103+
echo "::error::The RELEASE_FORK_TOKEN secret is not set."
104+
echo "Set it to a token with write access to ${{ vars.RELEASE_FORK_REPOSITORY }} and permission to create pull requests against this repository."
105+
exit 1
106+
fi
107+
echo "Changelog pull request branch will be pushed to fork: ${{ vars.RELEASE_FORK_REPOSITORY }}"
108+
33109
build:
34110
runs-on: ubuntu-latest
35111
timeout-minutes: 50
112+
needs: verify-preconditions
36113

37114
steps:
38115
- name: Set up JDK
@@ -181,24 +258,12 @@ jobs:
181258
echo 'release_major_tag: ${{ github.event.inputs.release-major-tag }}' >> release-description.yaml
182259
echo 'release_latest_tag: ${{ github.event.inputs.release-latest-tag }}' >> release-description.yaml
183260
184-
- name: Create tag
185-
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
186-
with:
187-
github-token: ${{ github.TOKEN }}
188-
script: |
189-
github.rest.git.createRef({
190-
owner: context.repo.owner,
191-
repo: context.repo.repo,
192-
ref: 'refs/tags/${{ env.version }}',
193-
sha: context.sha
194-
})
195-
196261
- name: Draft release
197262
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v1
198263
with:
199264
draft: true
200265
name: '${{ env.version }}'
201-
tag_name: 'refs/tags/${{ env.version }}'
266+
tag_name: '${{ env.version }}'
202267
files: |
203268
release-description.yaml
204269
@@ -227,17 +292,11 @@ jobs:
227292
- name: Generate Changelog
228293
run: release/script/changelog/generate-changelog.sh ${{ needs.promote.outputs.version }}
229294

230-
- name: GitHub App token
231-
id: github_app_token
232-
uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2
233-
with:
234-
app_id: ${{ secrets.APP_ID }}
235-
private_key: ${{ secrets.APP_PRIVATE_KEY }}
236-
237295
- name: Create Pull Request
238296
uses: peter-evans/create-pull-request@c5a7806660adbe173f04e3e038b0ccdcd758773c # v6
239297
with:
240-
token: ${{ steps.github_app_token.outputs.token }}
298+
token: ${{ secrets.RELEASE_FORK_TOKEN }}
299+
push-to-fork: ${{ vars.RELEASE_FORK_REPOSITORY }}
241300
add-paths: |
242301
release/release-notes/data-prepper.change-log-${{ needs.promote.outputs.version }}.md
243302
commit-message: 'Add Data Prepper ${{ needs.promote.outputs.version }} changelog.'

‎release/README.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,82 @@ It is not a project and instead holds release notes and change logs for Data Pre
99
## Performing a Release
1010

1111
See [RELEASING.md](../RELEASING.md) for instructions to follow for any release.
12+
13+
## Configuring the GitHub repo
14+
15+
Running a release requires both AWS staging resources and a number of GitHub Actions secrets and variables.
16+
The release workflows ([Prepare Release Branch](../.github/workflows/release-prepare-branch.yml) and
17+
[Release Artifacts](../.github/workflows/release.yml)) read these to build and upload the artifacts and to
18+
open their pull requests (PRs).
19+
20+
### Deploy the AWS staging resources
21+
22+
The release build uploads archives and Maven artifacts to AWS S3 and publishes Docker images to a staging ECR
23+
repository. These resources, and the IAM role that GitHub Actions assumes to access them, are provisioned by
24+
the [staging-resources-cdk](staging-resources-cdk/README.md) project. Follow that project's README to install
25+
the CDK and deploy the stacks before running a release.
26+
27+
The CDK deployment provides the values you will use for the `RELEASE_IAM_ROLE`, `ARCHIVES_BUCKET_NAME`,
28+
`ARCHIVES_PUBLIC_URL`, and `ECR_REPOSITORY_URL` secrets described below.
29+
30+
### Configure a fork for pull requests
31+
32+
The OpenSearch project does not permit this repository's workflows to push branches and open PRs
33+
against itself. Instead, the release workflows push their PR branches to a fork and open the PR from that fork
34+
back against this repository. This applies to both the release preparation PR and the changelog PR.
35+
36+
To run a release build, you must have a fork of Data Prepper used for staging these changes and PRs.
37+
This must be an actual GitHub fork of the repository you are releasing, because GitHub only permits
38+
cross-repository PRs between repositories in the same fork network.
39+
40+
The fork is identified by the `RELEASE_FORK_REPOSITORY` variable, and the token used to push to it and open
41+
the PR is the `RELEASE_FORK_TOKEN` secret. `RELEASE_FORK_TOKEN` must be a **classic** personal access token
42+
(PAT) meeting the following requirements.
43+
44+
* It is a classic PAT. A fine-grained token does not work because the single token must act on two
45+
repositories (the fork it pushes to and this repository it opens the PR against).
46+
* Its owner has write access to the fork named in `RELEASE_FORK_REPOSITORY`.
47+
* It has the `public_repo` scope (or `repo` if the repository is private).
48+
* It has the `workflow` scope. This is required because the pushed branch may include changes to files
49+
under `.github/workflows`, and GitHub rejects such pushes from a token without it.
50+
51+
Because a classic PAT acts as its owner across all of their repositories, use a dedicated release or bot
52+
account to own this token rather than a maintainer's personal account, and set a short expiration so the
53+
token is rotated regularly.
54+
55+
### Secrets and variables
56+
57+
Configure the following on the repository that runs the release workflows. Create variables as
58+
[repository variables](https://docs.github.com/en/actions/learn-github-actions/variables) and secrets as
59+
[repository secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets).
60+
61+
| Name | Type | Source | Description |
62+
| ---- | ---- | ------ | ----------- |
63+
| `RELEASE_FORK_REPOSITORY` | Variable | You | The `owner/repo` of the fork the PR branches are pushed to, for example `opensearch-ci-bot/data-prepper`. |
64+
| `RELEASE_FORK_TOKEN` | Secret | You | A classic PAT used to push the PR branch to the fork and open the PR. See the requirements above. |
65+
| `RELEASE_IAM_ROLE` | Secret | CDK | The ARN of the IAM role GitHub Actions assumes to access the staging resources. |
66+
| `ARCHIVES_BUCKET_NAME` | Secret | CDK | The name of the S3 bucket the release archives and Maven artifacts are uploaded to. |
67+
| `ARCHIVES_PUBLIC_URL` | Secret | CDK | The public base URL the uploaded archives are served from, used by the tarball smoke tests. |
68+
| `ECR_REPOSITORY_URL` | Secret | CDK | The URL of the staging ECR repository the Docker images are pushed to. |
69+
70+
## Testing release changes
71+
72+
You can test release changes by running the release workflows from your own fork of Data Prepper.
73+
The release workflows always open their PRs against the repository that runs them, so when you run them
74+
from your fork, the PRs target your fork.
75+
76+
To set this up:
77+
* Run the workflows from your fork of Data Prepper (for example `your-user/data-prepper`).
78+
* Create a second fork of your fork in an account or organization you control (for example
79+
`your-test-org/data-prepper`). This second fork is the head repository that the PR branches are pushed to.
80+
It is required because GitHub does not allow a PR where the head and base are the same repository.
81+
Note that GitHub permits only one fork per account, so the second fork must live in a different account
82+
or organization; both forks remain in the same fork network, which is what allows the cross-repository PR.
83+
* Set `RELEASE_FORK_REPOSITORY` and `RELEASE_FORK_TOKEN` on your fork as described in
84+
[Configuring the GitHub repo](#configuring-the-github-repo), pointing `RELEASE_FORK_REPOSITORY` at the
85+
second fork.
86+
87+
To test, you must also make a few manual modifications that you will not check in.
88+
* Update the `get_approvers` step with `minimum-approvals: 1`. Otherwise, you will be blocked in the promote step.
89+
* Change your `CODEOWNERS` file to have only your user. Without this, the promote step will try to assign this to users who are not in your fork and then fail.
90+
* You should consider updating the "Build Jar Files" step to only `assemble` instead of `build` to speed up your tests. `./gradlew --parallel --max-workers 2 assemble`

0 commit comments

Comments
 (0)