Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
173 changes: 173 additions & 0 deletions .github/workflows/create-test-mirror-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
name: Create test image mirror PR

on:
workflow_dispatch:
inputs:
pr_number:
description: "PR number in dd-trace-java-docker-build (e.g. 123)"
required: true

jobs:
create-test-mirror-pr:
runs-on: ubuntu-latest
permissions:
id-token: write # Required for OIDC token federation
contents: read
steps:
- uses: DataDog/dd-octo-sts-action@acaa02eee7e3bb0839e4272dacb37b8f3b58ba80 # v1.0.3
id: octo-sts
with:
scope: DataDog/images
policy: dd-trace-java-docker-build.update-mirror

- name: Checkout DataDog/images
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: DataDog/images
token: ${{ steps.octo-sts.outputs.token }}

- name: Capture images HEAD SHA
id: images-head
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"

- name: Install crane
run: |
CRANE_VERSION="0.20.2"
curl -fsSL "https://github.com/google/go-containerregistry/releases/download/v${CRANE_VERSION}/go-containerregistry_Linux_x86_64.tar.gz" -o crane.tar.gz
tar -xzf crane.tar.gz crane
sudo mv crane /usr/local/bin/crane
rm crane.tar.gz

- name: Resolve digests and update mirror files
id: update-mirror
env:
PR_NUMBER: ${{ github.event.inputs.pr_number }}
run: |
python3 - <<'PYEOF'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ question: ‏About this step, what motivated you to switch from bash to python as you had a working version in bash previously?

🎯 suggestion: ‏It might be easier to run a script that is versioned in the repository rather than adding it inline into yml? 🤷 And as they are similar scripts in the update-mirror-digests workflow, you can reuse it there too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ question: ‏About this step, what motivated you to switch from bash to python as you had a working version in bash previously?

No good reason - I started with trying to bring the bash over, but found it easier to format and test out python with Claude... I think I ended up over-engineering with python though, so will switch back to bash!

🎯 suggestion: ‏It might be easier to run a script that is versioned in the repository rather than adding it inline into yml? 🤷 And as they are similar scripts in the update-mirror-digests workflow, you can reuse it there too

Hmm that could make the workflows more readable too 🤔

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so will switch back to bash!

make the workflows more readable

Yeah, think about what could be the simplest bricks to be reused by the workflows.
Ideally, if we could run them outside the workflow (when needed or to test them) like you did for the pin system test script, that would be ideal.

About python or bash, whatever. Just pick one that work well for the task and that will be easy to maintain by the team :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 8e3e105, I split the logic into 3 scripts:

The create-test-mirror-pr.yml workflow calls create-test-mirror-entries.sh which either adds test image entries or updates the existing test image digests in DataDog/images.

The update-mirror-digests.yml workflow calls update-ci-image-digests.sh which updates the existing ci-* image digests.

Both workflows source get-image-digests.sh which uses crane to get the latest digests for all CI variants in ghcr.io, given the prefix ci- or PRNUM_merge-.

The scripts were successful locally!

import subprocess, re, os

SOURCE_REPO = "ghcr.io/datadog/dd-trace-java-docker-build"
VARIANTS = [
"base", "7", "8", "11", "17", "21", "25", "tip",
"zulu8", "zulu11", "oracle8", "ibm8",
"semeru8", "semeru11", "semeru17",
"graalvm17", "graalvm21", "graalvm25",
]

pr_number = os.environ["PR_NUMBER"]
if not pr_number.isdigit():
raise ValueError(f"PR_NUMBER must be numeric, got: {pr_number!r}")

prefix = f"{pr_number}_merge-"
print(f"Resolving digests for prefix: {prefix!r}")

digests = {}
for variant in VARIANTS:
tag = f"{prefix}{variant}"
result = subprocess.run(
["crane", "digest", f"{SOURCE_REPO}:{tag}"],
capture_output=True, text=True, check=True,
)
digest = result.stdout.strip()
digests[variant] = digest
print(f" {tag}: {digest}")

# Check whether entries already exist in mirror.yaml (use base as sentinel)
with open("mirror.yaml", "r") as f:
yaml_content = f.read()

entries_exist = f"{SOURCE_REPO}:{prefix}base" in yaml_content
mode = "update" if entries_exist else "add"
print(f"\nMode: {mode} ({'entries exist, updating digests only' if entries_exist else 'no entries found, adding new entries'})")

github_output = os.environ.get("GITHUB_OUTPUT", "")
if github_output:
with open(github_output, "a") as f:
f.write(f"mode={mode}\n")

if mode == "add":
yaml_entries = []
for variant in VARIANTS:
tag = f"{prefix}{variant}"
source = f"{SOURCE_REPO}:{tag}"
yaml_entries.append(
f' - source: "{source}"\n'
f' dest:\n'
f' repo: "dd-trace-java-docker-build"\n'
f' tag: "{tag}"\n'
f' replication_target: ""\n'
)
with open("mirror.yaml", "a") as f:
f.write("".join(yaml_entries))
print(f"Appended {len(yaml_entries)} entries to mirror.yaml")

# Always update mirror.lock.yaml: replace digest in-place if entry exists, append if not
with open("mirror.lock.yaml", "r") as f:
lock_content = f.read()

for variant in VARIANTS:
tag = f"{prefix}{variant}"
source = f"{SOURCE_REPO}:{tag}"
digest = digests[variant]
pattern = rf"( - source: {re.escape(source)}\n digest: )sha256:[a-f0-9]+"
if re.search(pattern, lock_content):
lock_content = re.sub(pattern, rf"\g<1>{digest}", lock_content)
print(f"Updated mirror.lock.yaml: {tag}")
else:
lock_content = lock_content.rstrip("\n") + "\n"
lock_content += f" - source: {source}\n digest: {digest}\n"
print(f"Appended to mirror.lock.yaml: {tag}")

with open("mirror.lock.yaml", "w") as f:
f.write(lock_content)

PYEOF

- name: Define branch name
id: define-branch
run: echo "branch=ci/add-dd-trace-java-docker-build-test-images-pr${{ github.event.inputs.pr_number }}" >> "$GITHUB_OUTPUT"

- name: Commit changes
id: create-commit
env:
PR_NUMBER: ${{ github.event.inputs.pr_number }}
MODE: ${{ steps.update-mirror.outputs.mode }}
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add mirror.yaml mirror.lock.yaml
if [[ "$MODE" == "update" ]]; then
git commit -m "chore: Update dd-trace-java-docker-build test image digests for PR #${PR_NUMBER}"
else
git commit -m "chore: Add dd-trace-java-docker-build test images for PR #${PR_NUMBER}"
fi
echo "commit=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"

- name: Push changes
uses: DataDog/commit-headless@05d7b7ee023e2c7d01c47832d420c2503cd416f3 # action/v2.0.3
with:
token: "${{ steps.octo-sts.outputs.token }}"
branch: "${{ steps.define-branch.outputs.branch }}"
head-sha: "${{ steps.images-head.outputs.sha }}"
create-branch: true
command: push
commits: "${{ steps.create-commit.outputs.commit }}"

- name: Create or identify pull request
env:
GH_TOKEN: ${{ steps.octo-sts.outputs.token }}
PR_NUMBER: ${{ github.event.inputs.pr_number }}
run: |
BRANCH="${{ steps.define-branch.outputs.branch }}"
EXISTING_PR=$(gh pr list --repo DataDog/images --head "$BRANCH" --json url -q '.[0].url' 2>/dev/null || true)
if [[ -n "$EXISTING_PR" ]]; then
echo "PR already exists: $EXISTING_PR"
else
gh pr create \
--repo DataDog/images \
--draft \
--title "Add dd-trace-java-docker-build test images for PR #${PR_NUMBER}" \
--base master \
--head "$BRANCH" \
--body "Adds mirror entries for \`${PR_NUMBER}_merge-*\` test images from DataDog/dd-trace-java-docker-build#${PR_NUMBER}. To use in dd-trace-java CI, set \`TESTER_IMAGE_VERSION_PREFIX: \"${PR_NUMBER}_merge-\"\` in \`.gitlab-ci.yml\`."
fi
Comment thread
sarahchen6 marked this conversation as resolved.
Outdated
2 changes: 1 addition & 1 deletion .github/workflows/docker-tag.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Tag new images version
name: Tag new images version # triggers update-mirror-digests workflow
on:
schedule:
# Quarterly schedule, roughly aligned with JDK CPU
Expand Down
167 changes: 167 additions & 0 deletions .github/workflows/update-mirror-digests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
name: Update mirror digests for ci-* images

on:
workflow_run:
workflows: ["Tag new images version"]
types: [completed]
workflow_dispatch:

jobs:
update-mirror-digests:
if: ${{ github.event_name == 'workflow_dispatch' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
permissions:
id-token: write # Required for OIDC token federation
contents: read
steps:
- uses: DataDog/dd-octo-sts-action@acaa02eee7e3bb0839e4272dacb37b8f3b58ba80 # v1.0.3
id: octo-sts
with:
scope: DataDog/images
policy: dd-trace-java-docker-build.update-mirror

- name: Checkout DataDog/images
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: DataDog/images
token: ${{ steps.octo-sts.outputs.token }}

- name: Capture images HEAD SHA
id: images-head
run: echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"

- name: Install crane
run: |
CRANE_VERSION="0.20.2"
curl -fsSL "https://github.com/google/go-containerregistry/releases/download/v${CRANE_VERSION}/go-containerregistry_Linux_x86_64.tar.gz" -o crane.tar.gz
tar -xzf crane.tar.gz crane
sudo mv crane /usr/local/bin/crane
rm crane.tar.gz

- name: Get baseline digest for ci-base image
id: baseline
run: |
BASELINE=$(awk '/source:.*dd-trace-java-docker-build:ci-base/{found=1; next} found && /digest:/{print $2; exit}' mirror.lock.yaml || true)
echo "digest=${BASELINE}" >> "$GITHUB_OUTPUT"
echo "Baseline ci-base digest: ${BASELINE:-<none found>}"

- name: Wait for new ci-base image to be published
run: |
BASELINE="${{ steps.baseline.outputs.digest }}"
DEADLINE=$((SECONDS + 1800)) # 30 min timeout
echo "Waiting for ci-base digest to differ from: ${BASELINE:-<none>}"
while [[ $SECONDS -lt $DEADLINE ]]; do
CURRENT=$(crane digest ghcr.io/datadog/dd-trace-java-docker-build:ci-base 2>/dev/null || true)
if [[ -n "$CURRENT" && "$CURRENT" != "$BASELINE" ]]; then
echo "New ci-base digest detected: $CURRENT"
exit 0
fi
echo "No change yet (current: ${CURRENT:-unavailable}), retrying in 60s..."
sleep 60
done
echo "::error::Timeout after 30 minutes: ci-base digest did not change"
exit 1

- name: Resolve digests and update mirror files
run: |
python3 - <<'PYEOF'
import subprocess, re

SOURCE_REPO = "ghcr.io/datadog/dd-trace-java-docker-build"
VARIANTS = [
"base", "7", "8", "11", "17", "21", "25", "tip",
"zulu8", "zulu11", "oracle8", "ibm8",
"semeru8", "semeru11", "semeru17",
"graalvm17", "graalvm21", "graalvm25",
]

# Verify all ci-* entries are already present in both files before proceeding
with open("mirror.yaml", "r") as f:
yaml_content = f.read()
with open("mirror.lock.yaml", "r") as f:
lock_content = f.read()

missing_yaml = [v for v in VARIANTS if f"{SOURCE_REPO}:ci-{v}" not in yaml_content]
missing_lock = [v for v in VARIANTS if f"{SOURCE_REPO}:ci-{v}" not in lock_content]
if missing_yaml or missing_lock:
if missing_yaml:
print(f"::error::ci-* entries missing from mirror.yaml: {missing_yaml}")
if missing_lock:
print(f"::error::ci-* entries missing from mirror.lock.yaml: {missing_lock}")
print("Bootstrap the ci-* entries manually before running this workflow.")
raise SystemExit(1)

print("Resolving digests for ci-* variants...")
digests = {}
for variant in VARIANTS:
tag = f"ci-{variant}"
result = subprocess.run(
["crane", "digest", f"{SOURCE_REPO}:{tag}"],
capture_output=True, text=True, check=True,
)
digest = result.stdout.strip()
digests[variant] = digest
print(f" {tag}: {digest}")

# Update existing digest entries in mirror.lock.yaml in-place
for variant in VARIANTS:
tag = f"ci-{variant}"
source = f"{SOURCE_REPO}:{tag}"
digest = digests[variant]
pattern = rf"( - source: {re.escape(source)}\n digest: )sha256:[a-f0-9]+"
lock_content = re.sub(pattern, rf"\g<1>{digest}", lock_content)
print(f"Updated mirror.lock.yaml: {tag}")

with open("mirror.lock.yaml", "w") as f:
f.write(lock_content)

PYEOF

- name: Check for changes
id: check-changes
run: |
if [[ -z "$(git status -s)" ]]; then
echo "No changes to commit."
echo "commit_changes=false" >> "$GITHUB_OUTPUT"
else
echo "commit_changes=true" >> "$GITHUB_OUTPUT"
fi

- name: Define branch name
if: steps.check-changes.outputs.commit_changes == 'true'
id: define-branch
run: echo "branch=ci/update-dd-trace-java-docker-build-ci-digests-$(date +'%Y%m%d')" >> "$GITHUB_OUTPUT"

- name: Commit changes
if: steps.check-changes.outputs.commit_changes == 'true'
id: create-commit
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git add mirror.yaml mirror.lock.yaml
git commit -m "chore: Update dd-trace-java-docker-build ci-* image digests"
echo "commit=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"

- name: Push changes
if: steps.check-changes.outputs.commit_changes == 'true'
uses: DataDog/commit-headless@05d7b7ee023e2c7d01c47832d420c2503cd416f3 # action/v2.0.3
with:
token: "${{ steps.octo-sts.outputs.token }}"
branch: "${{ steps.define-branch.outputs.branch }}"
head-sha: "${{ steps.images-head.outputs.sha }}"
create-branch: true
command: push
commits: "${{ steps.create-commit.outputs.commit }}"

- name: Create pull request
if: steps.check-changes.outputs.commit_changes == 'true'
env:
GH_TOKEN: ${{ steps.octo-sts.outputs.token }}
run: |
gh pr create \
--repo DataDog/images \
--draft \
--title "Update dd-trace-java-docker-build ci-* image digests" \
--base master \
--head "${{ steps.define-branch.outputs.branch }}" \
--body "Automated digest update for \`dd-trace-java-docker-build\` \`ci-*\` images after tagging."
25 changes: 19 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# dd-trace-java-docker-build

This repository holds Docker images for continuous integration jobs at [dd-trace-java](https://github.com/datadog/dd-trace-java).
This repository holds the original Docker images for continuous integration jobs in [dd-trace-java](https://github.com/datadog/dd-trace-java). The images built here are mirrored into `registry.ddbuild.io` before use in `dd-trace-java` CI. This is to ensure that all CI images are properly signed. See [DataDog/images image-mirroring](https://github.com/DataDog/images#image-mirroring) for more details.

## Usage

Pre-built images are available in [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build).
Pre-built images are available in the [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build).

Image variants are available on a per JDK basis:
- The `base` variant and its aliases, `8`, `11`, `17`, `21`, `25`, and `tip`, contain the base Eclipse Temurin JDK 8, 11, 17, 21, 25, and tip JDK version releases,
- The `zulu8`, `zulu11`, `oracle8`, `ibm8`, `semeru8`, `semeru11`, `semeru17`, `graalvm17`, `graalvm21`, and `graalvm25` variants all contain the base JDKs in addition to the specific JDK from their name,
- The `latest` variant contains the base JDKs and all the above specific JDKs.
- The `base` variant and its aliases`8`, `11`, `17`, `21`, `25`, and `tip`contain the base Eclipse Temurin JDK 8, 11, 17, 21, 25, and tip JDK version releases.
- The `zulu8`, `zulu11`, `oracle8`, `ibm8`, `semeru8`, `semeru11`, `semeru17`, `graalvm17`, `graalvm21`, and `graalvm25` variants all contain the base JDKs in addition to the specific JDK from their name.
- The `latest` variant contains the base JDKs and all of the specific JDKs above.

Images are tagged via the [Tag new images version](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/docker-tag.yml) workflow. This workflow tags the latest images built from the specified branch with a `ci-` prefix. It runs quarterly on `master` but can also be triggered manually as needed.
Images are tagged via the [Tag new images version](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/docker-tag.yml) workflow. This workflow tags the latest images built from the specified branch with a `ci-` prefix and triggers the [Update mirror digests for ci-* images](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/update-mirror-digests.yml) workflow which automatically creates a PR that updates the pinned `ci-*` image digests in `DataDog/images`. It runs quarterly on `master` but can also be triggered manually as needed.

## Development

Expand All @@ -26,3 +26,16 @@ And then check the built images:
```bash
./build --test
```

## Testing
Comment thread
sarahchen6 marked this conversation as resolved.

Images are built per PR for ease in testing. These test images are prefixed with `N_merge-`, where N is the PR number. See the [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build) for examples.

To test these images in `dd-trace-java` CI:

1. Open a PR in [DataDog/dd-trace-java-docker-build](https://github.com/DataDog/dd-trace-java-docker-build) with the changes you want to test. Let's say these changes are made in PR #123 ([example](https://github.com/DataDog/dd-trace-java-docker-build/pull/123)).
2. Run the [Create test image mirror PR](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/create-test-mirror-pr.yml) workflow with `pr_number=123`. This automatically opens a PR in [DataDog/images](https://github.com/DataDog/images) that adds mirror entries for the `123_merge-*` test images. The PR should be automatically merged by the `dd-prapprover` bot.
Comment thread
PerfectSlayer marked this conversation as resolved.
Outdated
3. Open a PR in [DataDog/dd-trace-java](https://github.com/DataDog/dd-trace-java) that sets `BUILDER_IMAGE_VERSION_PREFIX: "123_merge-"` in `.gitlab-ci.yml`. Here, you can check your test images with `DataDog/dd-trace-java` CI.
4. Every time you want to test changes made in PR #123, ensure the test image SHAs in `DataDog/images` are updated. This should be done by running the [Create test image mirror PR](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/create-test-mirror-pr.yml) workflow with `pr_number=123`.
5. When the test images look good and `DataDog/dd-trace-java` CI is green, merge your `DataDog/dd-trace-java-docker-build` PR #123, close the test `DataDog/dd-trace-java` PR, and **remove the test images from the `DataDog/images` repo**.
6. Finally, run the [Tag new images version](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/docker-tag.yml) workflow. The [Update mirror digests for ci-* images](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/update-mirror-digests.yml) workflow will automatically open a PR in `DataDog/images`, updating the pinned `ci-*` digests.
Comment thread
PerfectSlayer marked this conversation as resolved.
Outdated