Skip to content

fix: clean up untagged ECR images after overwrite_only_existing mirror#245

Merged
lolgab merged 5 commits into
masterfrom
delete-untagged-ecr
Jun 3, 2026
Merged

fix: clean up untagged ECR images after overwrite_only_existing mirror#245
lolgab merged 5 commits into
masterfrom
delete-untagged-ecr

Conversation

@lolgab
Copy link
Copy Markdown
Contributor

@lolgab lolgab commented Jun 3, 2026

Summary

  • After a successful push with overwrite_only_existing, list all untagged images in the ECR repository via list-images --filter tagStatus=UNTAGGED and delete them with batch-delete-image
  • Avoids deleting digests that still have other tags (e.g. latest sharing the same digest as the overwritten tag)
  • Also cleans up any previously accumulated untagged images in the repository

Test plan

  • CI pack_and_validate passes
  • Run mirror job with overwrite_only_existing: true on a repo/tag that already exists and confirm no untagged digests remain
  • Verify a digest shared by multiple tags is not deleted when only one tag is overwritten

When overwriting an existing tag, ECR leaves the old manifest untagged.
Remove it with batch-delete-image so hourly mirrors do not accumulate storage.
@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented Jun 3, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes.

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the ECR mirroring job to delete the previous untagged image digest from ECR after overwriting an existing image tag. A critical issue was identified in the review: if the newly pushed image has the same digest as the old image, the deletion step will mistakenly delete the newly pushed image, leaving the repository empty. A code suggestion was provided to verify that the image digest has actually changed before performing the deletion.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/jobs/mirror_to_ecr.yml Outdated
lolgab added 2 commits June 3, 2026 10:04
Collapse IMAGE_EXISTS/SHOULD_MIRROR into direct conditions without
changing mirror, skip, fail, or digest-delete behavior.
After push, compare the new image digest with the pre-push digest and
only batch-delete the old digest when they differ, avoiding removal of
the image that still carries the tag.
@lolgab lolgab marked this pull request as ready for review June 3, 2026 08:10
@lolgab lolgab changed the title Delete untagged ECR image after overwrite_only_existing mirror feature: Delete untagged ECR image after overwrite_only_existing mirror Jun 3, 2026
Copy link
Copy Markdown

@codacy-production codacy-production Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

While this PR successfully implements the logic to capture and delete old image digests after an overwrite, the current implementation carries a risk of unintended data loss. Deleting an ECR image by digest removes all tags associated with that digest, which may include versioned or stable tags that should be preserved.

Additionally, the PR lacks automated verification (e.g., bats tests) for the YAML job logic, and the script relies on brittle string matching for AWS CLI outputs. Although the code is technically 'up to standards' according to Codacy, addressing the destructive deletion logic is critical before merging.

About this PR

  • The PR lacks automated test files (e.g., bats or shell script tests) to verify the logic changes within the YAML job definition. Adding unit tests for these shell blocks would prevent future regressions in mirroring logic.
  • The script explicitly checks for 'None' strings from AWS CLI output. This is brittle; if the output format is changed or if the CLI version behaves differently in certain environments, the logic may fail. Consider using JSON output and parsing with a tool like jq for more robust results.

Test suggestions

  • Missing: Successful overwrite of existing tag results in deletion of the previous digest.
  • Missing: Job fails as expected when overwrite_only_existing is true but the tag is missing.
  • Missing: Pushing an identical image (same digest) does not trigger a deletion attempt.
  • Missing: Standard mirroring (not an overwrite) functions without attempting to delete digests.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Missing: Successful overwrite of existing tag results in deletion of the previous digest.
2. Missing: Job fails as expected when overwrite_only_existing is true but the tag is missing.
3. Missing: Pushing an identical image (same digest) does not trigger a deletion attempt.
4. Missing: Standard mirroring (not an overwrite) functions without attempting to delete digests.

TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback

Comment thread src/jobs/mirror_to_ecr.yml Outdated
[[ -z "${NEW_IMAGE_DIGEST}" || "${NEW_IMAGE_DIGEST}" == "None" ]] && NEW_IMAGE_DIGEST=""
if [[ -n "${NEW_IMAGE_DIGEST}" ]] && [[ "${OLD_IMAGE_DIGEST}" != "${NEW_IMAGE_DIGEST}" ]]; then
echo "Deleting previous untagged image ${OLD_IMAGE_DIGEST} from ECR..."
aws ecr batch-delete-image --repository-name=${MIRROR_NAME} --image-ids imageDigest=${OLD_IMAGE_DIGEST}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 HIGH RISK

Deleting the image by digest removes all tags associated with that image, not just the one being overwritten. This can lead to unintended data loss if the same image is referenced by multiple tags (e.g., 'v1.0.1' and 'latest'). To ensure only truly 'untagged' images are removed, verify that no other tags remain for that digest before deletion.

Try updating the deletion logic to check if OLD_IMAGE_DIGEST still has associated tags by querying imageDetails[0].imageTags via aws ecr describe-images. Only execute batch-delete-image if the tag list is empty or returns 'None'.

MIRROR_NAME="${SRC_NAME}"
fi

MIRROR_NAME="${MIRROR_NAME:-${SRC_NAME}}"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Suggestion: Verify that '<< parameters.mirror_name >>' evaluates to an empty string when not provided so that '${MIRROR_NAME:-${SRC_NAME}}' correctly defaults to the source name.

Comment thread src/jobs/mirror_to_ecr.yml Outdated
SHOULD_MIRROR=true
elif << parameters.force >> || << parameters.overwrite_only_existing >> ; then
SHOULD_MIRROR=true
OLD_IMAGE_DIGEST=$(aws ecr describe-images --repository-name=${MIRROR_NAME} --image-ids=imageTag=${MIRROR_TAG} --query 'imageDetails[0].imageDigest' --output text 2>/dev/null || true)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Suggestion: Quote the variables to prevent potential shell expansion issues.

Suggested change
OLD_IMAGE_DIGEST=$(aws ecr describe-images --repository-name=${MIRROR_NAME} --image-ids=imageTag=${MIRROR_TAG} --query 'imageDetails[0].imageDigest' --output text 2>/dev/null || true)
OLD_IMAGE_DIGEST=$(aws ecr describe-images --repository-name="${MIRROR_NAME}" --image-ids=imageTag="${MIRROR_TAG}" --query 'imageDetails[0].imageDigest' --output text 2>/dev/null || true)

Replace digest-targeted deletion with list-images tagStatus=UNTAGGED
cleanup so shared-tag digests are never removed unintentionally.
@lolgab lolgab changed the title feature: Delete untagged ECR image after overwrite_only_existing mirror fix: clean up untagged ECR images after overwrite_only_existing mirror Jun 3, 2026
@lolgab lolgab merged commit b36f2c1 into master Jun 3, 2026
6 checks passed
@lolgab lolgab deleted the delete-untagged-ecr branch June 3, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants