Skip to content

fix(ci): get latest created tag for catalog remote tests #576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ajaskolski merged 3 commits into from
Nov 13, 2025
Merged

fix(ci): get latest created tag for catalog remote tests #576

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
088d05a
fix(ci): get latest tag for catalog remote tests
ajaskolski Nov 13, 2025
4ad1e27
simplify code logic
ajaskolski Nov 13, 2025
2d95e71
fix
ajaskolski Nov 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions .github/workflows/pull-request-main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,17 +79,51 @@ jobs:
id-token: write
contents: read
actions: read
env:
CATALOG_SERVICE_IMAGE: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/op-catalog-service:latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Configure AWS credentials for ECR
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: ${{ secrets.ECR_READ_ROLE_ARN }}
aws-region: ${{ secrets.AWS_REGION }}

- name: Get latest catalog service tag from ECR
id: get-catalog-tag
run: |
# Fetch the latest semantic version tag from ECR
# Sort images by push date (latest first) and find the first semver tag
LATEST_TAG=$(aws ecr describe-images \

@DimitriosNaikopoulos DimitriosNaikopoulos Nov 13, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this will return images from older to latest. Also not sure what will happen when there are too many (eg will we use pagination?)

You could use something like this to sort then based on push date in reverse (latest first)

aws ecr describe-images --repository-name op-catalog-service \
--query 'reverse(sort_by(imageDetails,& imagePushedAt))'

@ajaskolski ajaskolski Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually good idea there I will update it like that ! thx, updated !

@ajaskolski ajaskolski Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok now it works, simplified a bit.

--repository-name op-catalog-service \
--region ${{ secrets.AWS_REGION }} \
--registry-id ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }} \
--query 'reverse(sort_by(imageDetails,& imagePushedAt))' \
--output json | \
jq -r '
[.[] |
select(.imageTags != null) |
.imageTags[] |
select(test("^v[0-9]+\\.[0-9]+\\.[0-9]+$"))
] | first
')

if [[ -z "${LATEST_TAG}" ]] || [[ "${LATEST_TAG}" == "null" ]]; then

@graham-chainlink graham-chainlink Nov 13, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm i wonder if this approach of running integration test on latest docker image is the right appraoch? Shouldn we be building the docker image using the PR branch instead?

What if i updated or made a breaking change and modified the intergration tests, i would always get a failure here right?

@ajaskolski ajaskolski Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We spoke with James about it. So we own both products if we do introduce breaking changes we update upstream and downstream (cldf) in sync. You can test locally your breaking changes with local service version and update cldf when upstream service is released.

@ajaskolski ajaskolski Nov 13, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also if somebody make breaking changes here we should fail it imo, otherwise someone can update breaking version of the CLDF in projects importing it that won't work with the running service.

@graham-chainlink graham-chainlink Nov 13, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait i was confused, this makes sense!

echo "::error::Could not find a valid semantic version tag in ECR"
exit 1
fi

echo "Latest catalog service tag from ECR: ${LATEST_TAG}"
echo "tag=${LATEST_TAG}" >> "${GITHUB_OUTPUT}"

- name: Pull Catalog Service ECR Image
uses: smartcontractkit/.github/actions/pull-private-ecr-image@2f8f0baf38e46140c6a119eb551a56eaaabcc09e # pull-private-ecr-image@1.0.0
with:
aws-account-number: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}
aws-region: ${{ secrets.AWS_REGION }}
aws-role-arn: ${{ secrets.ECR_READ_ROLE_ARN }}
ecr-repository: "op-catalog-service"
image-tag: "latest"
image-tag: ${{ steps.get-catalog-tag.outputs.tag }}

- name: Run Catalog Remote Integration Tests
uses: smartcontractkit/.github/actions/ci-test-go@dfcba48f05933158428bce867d790e3d5a9baa6b # ci-test-go@1.1.0
Expand All @@ -98,6 +132,8 @@ jobs:
go-test-cmd: cd datastore/catalog/remote && go test -v -race -timeout 10m -gcflags=all=-d=checkptr=0 -coverprofile=../../../coverage.txt
use-go-cache: true
artifact-name: catalog-remote-tests
env:
CATALOG_SERVICE_IMAGE: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/op-catalog-service:${{ steps.get-catalog-tag.outputs.tag }}

sonarqube:
name: Sonar Scan
Expand Down
42 changes: 39 additions & 3 deletions .github/workflows/schedule-main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,51 @@ jobs:
id-token: write
contents: read
actions: read
env:
CATALOG_SERVICE_IMAGE: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/op-catalog-service:latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Configure AWS credentials for ECR
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: ${{ secrets.ECR_READ_ROLE_ARN }}
aws-region: ${{ secrets.AWS_REGION }}

- name: Get latest catalog service tag from ECR
id: get-catalog-tag
run: |
# Fetch the latest semantic version tag from ECR
# Sort images by push date (latest first) and find the first semver tag
LATEST_TAG=$(aws ecr describe-images \
--repository-name op-catalog-service \
--region ${{ secrets.AWS_REGION }} \
--registry-id ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }} \
--query 'reverse(sort_by(imageDetails,& imagePushedAt))' \
--output json | \
jq -r '
[.[] |
select(.imageTags != null) |
.imageTags[] |
select(test("^v[0-9]+\\.[0-9]+\\.[0-9]+$"))
] | first
')

if [[ -z "${LATEST_TAG}" ]] || [[ "${LATEST_TAG}" == "null" ]]; then
echo "::error::Could not find a valid semantic version tag in ECR"
exit 1
fi

echo "Latest catalog service tag from ECR: ${LATEST_TAG}"
echo "tag=${LATEST_TAG}" >> "${GITHUB_OUTPUT}"

- name: Pull Catalog Service ECR Image
uses: smartcontractkit/.github/actions/pull-private-ecr-image@2f8f0baf38e46140c6a119eb551a56eaaabcc09e # pull-private-ecr-image@1.0.0
with:
aws-account-number: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}
aws-region: ${{ secrets.AWS_REGION }}
aws-role-arn: ${{ secrets.ECR_READ_ROLE_ARN }}
ecr-repository: "op-catalog-service"
image-tag: "latest"
image-tag: ${{ steps.get-catalog-tag.outputs.tag }}

- name: Run Catalog Remote Integration Tests
uses: smartcontractkit/.github/actions/ci-test-go@dfcba48f05933158428bce867d790e3d5a9baa6b # ci-test-go@1.1.0
Expand All @@ -94,6 +128,8 @@ jobs:
go-test-cmd: cd datastore/catalog/remote && go test -v -race -timeout 10m -gcflags=all=-d=checkptr=0 -coverprofile=../../../coverage.txt
use-go-cache: true
artifact-name: catalog-remote-tests
env:
CATALOG_SERVICE_IMAGE: ${{ secrets.AWS_ACCOUNT_NUMBER_PROD }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com/op-catalog-service:${{ steps.get-catalog-tag.outputs.tag }}

sonarqube:
name: Sonar Scan
Expand Down
Loading