Skip to content

CmampTask7965_Fix_the_triggering_of_the_release_Docker_images_workflow #1096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
dremdem wants to merge 12 commits into
base: master
Choose a base branch
from
Draft

CmampTask7965_Fix_the_triggering_of_the_release_Docker_images_workflow #1096

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
806d920
Update the trigger event and token usage
Nov 10, 2025
ca000f5
Update comment
dremdem Nov 12, 2025
e464fa2
Testing
dremdem Nov 14, 2025
e72485d
Change token everythere
dremdem Nov 14, 2025
d3020f3
Force reviewer
dremdem Nov 14, 2025
4e5b56f
Branch naming changed for tesing
dremdem Nov 14, 2025
c30ff7d
Back to token
dremdem Nov 14, 2025
dea3d61
Fix perm
dremdem Nov 14, 2025
f402445
Fix perm
dremdem Nov 14, 2025
fa18b6f
Ssh
dremdem Nov 17, 2025
847db00
Merge branch 'master' into CmampTask7965_Fix_the_triggering_of_the_re…
dremdem Nov 17, 2025
11b8ddc
Debug
dremdem Nov 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions .github/workflows/common_dev_image_build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ permissions:
# This is required for actions/checkout.
contents: write
# This is required for pulling the Docker image from GHCR.
packages: read
packages: write
# This is required for GitHub App to create issues and PRs.
pull-requests: write
issues: write
Expand All @@ -28,6 +28,14 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Generate GitHub App token
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ vars.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}

# Pass AWS credentials via GH secrets. This is needed to pull the Docker
# image and in case the workflow needs to access AWS resources.
- name: Configure AWS credentials
Expand All @@ -46,7 +54,7 @@ jobs:
# This is needed to pull the Docker image.
- name: Login to GHCR
run: |
echo "${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}" \
echo "${{ secrets.GITHUB_TOKEN }}" \
| docker login ghcr.io -u ${{ github.actor }} --password-stdin

# Make everything accessible by any user to avoid permission errors.
Expand All @@ -66,7 +74,7 @@ jobs:
# make it a default behavior? For certain tests to pass, we need entire
# commit history of the repo including sub-modules.
fetch-depth: 0
token: ${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}
token: ${{ secrets.GITHUB_TOKEN }}

# To access modules in `amp` and `helpers_root`, make sure PYTHONPATH includes
# them, just as it's set in `setenv.sh`.
Expand Down Expand Up @@ -99,16 +107,19 @@ jobs:
# Setup GitHub CLI authentication for creating issues and PRs.
- name: Setup GitHub CLI
run: |
echo "${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}" \
echo "${{ secrets.GITHUB_TOKEN }}" \
| gh auth login --with-token

# Run the dev image build and test workflow.
- name: Run 'docker_build_test_dev_image' workflow
uses: lhotari/action-upterm@v1
env:
CSFY_AWS_ACCESS_KEY_ID: ${{ env.AWS_ACCESS_KEY_ID }}
CSFY_AWS_SECRET_ACCESS_KEY: ${{ env.AWS_SECRET_ACCESS_KEY }}
CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}
run: invoke docker_build_test_dev_image
GH_ACTION_ACCESS_TOKEN: ${{ steps.app-token.outputs.token }}
GH_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# TODO(Vlad): Reviewer for testing puprose, remove before merge.
# run: invoke docker_build_test_dev_image --reviewers=dremdem
24 changes: 13 additions & 11 deletions .github/workflows/common_dev_image_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
name: Common Dev Image Release
on:
workflow_call:
inputs:
container-dir-name:
description: 'Container directory name - directory where Dockerfile and changelog.txt are located.'
required: false
type: string
default: '.'
workflow_call:
inputs:
container-dir-name:
# Repo root or runnable dir to release the image from (must have devops/ and changelog.txt).
# Examples: '.' (repo root, default) or 'subdir_name' (runnable dir)
description: 'Container directory name - directory where Dockerfile and changelog.txt are located.'
required: false
type: string
default: '.'
Comment thread
dremdem marked this conversation as resolved.
env:
CSFY_CI: true
# CSFY_ECR_BASE_PATH: ${{ vars.CSFY_ECR_BASE_PATH }}
Expand Down Expand Up @@ -45,7 +47,7 @@ jobs:
# This is needed to pull the Docker image.
- name: Login to GHCR
run: |
echo "${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}" \
echo "${{ secrets.GITHUB_TOKEN }}" \
| docker login ghcr.io -u ${{ github.actor }} --password-stdin
Comment on lines 48 to 51
Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also could we see how the PR would like when it's filed by the GH app as well?

Copy link
Copy Markdown
Contributor Author

@dremdem dremdem Nov 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!
I tried to make and test another PR: #1099 and found:

However, the GITHUB_TOKEN can only access resources within the workflow's repository. If you need to access additional resources, such as resources in an organization or in another repository, you can use a GitHub App.

The question is: who is able to set up these variables and secrets in the helpers repo?

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which variables need to be set?

Copy link
Copy Markdown
Contributor Author

@dremdem dremdem Nov 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • .github/workflows/sprint_iteration.yml

Like in the:
https://github.com/causify-ai/csfy/blob/f7b4afa69bf24b6f4ffedd807bf1825fdf143e09/.github/workflows/sprint_iteration.yml#L25-L26

      - name: Generate GitHub App token
        uses: actions/create-github-app-token@v1
        id: app-token
        with:
          app-id: ${{ vars.GH_APP_ID }}
          private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
          owner: ${{ github.repository_owner }}

We need to set:

  • variable: GH_APP_ID
  • secret: GH_APP_PRIVATE_KEY

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see they are already set as ORG level secrets/vars so they should already be accessible by //helpers right?

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also try to test it on //csfy too

Copy link
Copy Markdown
Contributor Author

@dremdem dremdem Nov 14, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see they are already set as ORG level secrets/vars so they should already be accessible by //helpers right?

I tried to debug in in the ssh-session inside the GitHub workflow run, but they are empty:
the ssh-session step to see the variable and the secret:

helpers/.github/workflows/common_dev_image_build.yml

Lines 31 to 36 in 23192e7

- name: ssh
uses: lhotari/action-upterm@v1
env:
app-id: ${{ vars.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}

  • checked inside the ssh-session:
runner@runnervmw9dnm:~/work/helpers/helpers$ env | grep app
app-id=
runner@runnervmw9dnm:~/work/helpers/helpers$ env | grep private
private-key=
runner@runnervmw9dnm:~/work/helpers/helpers$

Probably need to set permission for these variable and secret on the repo level also.
https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets?utm_source=chatgpt.com#reviewing-access-to-organization-level-secrets

@heanhsok
Can you pls check it.

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They should be there now. I don't have perm to check that but I could replicate them in //helpers.
Cloud you try again?


# Make everything accessible by any user to avoid permission errors.
Expand All @@ -62,7 +64,7 @@ jobs:
# make it a default behavior? For certain tests to pass, we need entire
# commit history of the repo including sub-modules.
fetch-depth: 0
token: ${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}
token: ${{ secrets.GITHUB_TOKEN }}

# To access modules in `amp` and `helpers_root`, make sure PYTHONPATH includes
# them, just as it's set in `setenv.sh`.
Expand All @@ -89,8 +91,8 @@ jobs:
CSFY_AWS_SESSION_TOKEN: ${{ env.AWS_SESSION_TOKEN }}
CSFY_AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
CSFY_AWS_S3_BUCKET: ${{ vars.CSFY_AWS_S3_BUCKET }}
GH_ACTION_ACCESS_TOKEN: ${{ secrets.GH_ACTION_ACCESS_TOKEN || secrets.GITHUB_TOKEN }}
run: invoke docker_tag_push_dev_image
GH_ACTION_ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: invoke docker_tag_push_dev_image --container-dir-name="${{ inputs.container-dir-name }}"

# Generate release message.
- name: Generate release message
Expand Down
15 changes: 9 additions & 6 deletions .github/workflows/dev_image_release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
name: Dev image release
on:
# Trigger on a merged PR, with restrictions applied at the job level.
pull_request:
types: [closed]
Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this CI pipeline going to appear in all the closed PR?

Copy link
Copy Markdown
Contributor Author

@dremdem dremdem Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but there’s an if condition here:

helpers/.github/workflows/dev_image_release.yml

Lines 21 to 25 in 806d920

if: >
(
github.event.pull_request.merged == true
&& contains(github.event.pull_request.labels.*.name, 'Automated release')
) || github.event_name == 'workflow_dispatch'
)

So, the workflow will be triggered, but the job itself won’t run unless the condition is met.

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think let's not have it triggered and shown for non relevant PRs

Copy link
Copy Markdown
Contributor Author

@dremdem dremdem Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find a way to prevent the GitHub workflow from triggering when we merge a PR with the Automated build label.
The only solution I’ve found is to trigger on any merged or closed PR and then filter them out at the job level.

Another possible approach could be:

  • Keep this GitHub workflow as a manual run
  • Add an extra manual step in our pipeline to execute this workflow

Copy link
Copy Markdown
Contributor

@heanhsok heanhsok Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about triggered it on merged to master when the changelog.txt is changed?

We can do path filtering

https://github.com/causify-ai/csfy/blob/f7b4afa69bf24b6f4ffedd807bf1825fdf143e09/.github/workflows/platform-ci.yml#L4-L8

branches:
- master
# Run manually.
workflow_dispatch:
# Set up permissions for OIDC authentication.
Expand All @@ -14,12 +19,10 @@ concurrency:
jobs:
dev_image_release:
if: >
${{
(github.event_name == 'pull_request'
&& github.event.pull_request.merged == true
&& contains(github.event.pull_request.labels.*.name, 'Automated release')) ||
github.event_name == 'workflow_dispatch'
}}
(
github.event.pull_request.merged == true
&& contains(github.event.pull_request.labels.*.name, 'Automated release')
) || github.event_name == 'workflow_dispatch'
uses: ./.github/workflows/common_dev_image_release.yml
with:
container-dir-name: .
Expand Down
11 changes: 7 additions & 4 deletions helpers/lib_tasks_docker_release.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1700,7 +1700,9 @@ def docker_build_test_dev_image( # type: ignore
_LOG.info("Step 3: Creating branch with date-based name")
issue_prefix = hrecouti.get_repo_config().get_issue_prefix()
# Get current date in YYYYMMDD format.
today = datetime.date.today().strftime("%Y%m%d")
# TODO(Vlad): For testing, need revert back before merging.
today = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
# today = datetime.date.today().strftime("%Y%m%d")
branch_name = f"{issue_prefix}_Periodic_image_release_{today}"
_LOG.info("Branch name: %s", branch_name)
cmd = f"git checkout -b {branch_name}"
Expand All @@ -1723,9 +1725,10 @@ def docker_build_test_dev_image( # type: ignore
stage,
dev_version,
skip_tests=False,
fast_tests=True,
slow_tests=True,
superslow_tests=True,
# TODO(Vlad): Just for testing purposes, need to set to True before merging.
fast_tests=False,
slow_tests=False,
superslow_tests=False,
qa_tests=False,
)
# 6) Add changelog entry.
Expand Down
2 changes: 1 addition & 1 deletion repo_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ docker_info:
# Base name of the docker image, e.g., `helpers`.
docker_image_name: helpers
use_sibling_container_in_unit_tests: True
release_team: dev_system
release_team: dev_releasers

s3_bucket_info:
unit_test_bucket_name: s3://cryptokaizen-unit-test
Expand Down
Loading