Skip to content

Commit 1e6af16

Browse files
sarahchen6sarahchen6
committed
Add image mirroring support
1 parent 258638a commit 1e6af16

3 files changed

Lines changed: 136 additions & 32 deletions

File tree

‎.github/workflows/docker-tag.yml‎

Lines changed: 0 additions & 25 deletions
This file was deleted.

‎README.md‎

Lines changed: 27 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,15 @@
11
# dd-trace-java-docker-build
22

3-
This repository holds Docker images for continuous integration jobs at [dd-trace-java](https://github.com/datadog/dd-trace-java).
3+
This repository holds the original Docker images for continuous integration jobs in [dd-trace-java](https://github.com/datadog/dd-trace-java). The images built here are mirrored into `registry.ddbuild.io` before use in `dd-trace-java` CI. This is to ensure that all CI images are properly signed. See [DataDog/images image-mirroring](https://github.com/DataDog/images#image-mirroring) for more details.
44

55
## Usage
66

7-
Pre-built images are available in [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build).
7+
Pre-built images are available in the [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build).
88

99
Image variants are available on a per JDK basis:
10-
- The `base` variant and its aliases, `8`, `11`, `17`, `21`, `25`, and `tip`, contain the base Eclipse Temurin JDK 8, 11, 17, 21, 25, and tip JDK version releases,
11-
- The `zulu8`, `zulu11`, `oracle8`, `ibm8`, `semeru8`, `semeru11`, `semeru17`, `graalvm17`, `graalvm21`, and `graalvm25` variants all contain the base JDKs in addition to the specific JDK from their name,
12-
- The `latest` variant contains the base JDKs and all the above specific JDKs.
13-
14-
Images are tagged via the [Tag new images version](https://github.com/DataDog/dd-trace-java-docker-build/actions/workflows/docker-tag.yml) workflow. This workflow tags the latest images built from the specified branch using the `vYY.MM` format. It runs quarterly on `master` but can also be triggered manually as needed.
10+
- The `base` variant and its aliases — `8`, `11`, `17`, `21`, `25`, and `tip` — contain the base Eclipse Temurin JDK 8, 11, 17, 21, 25, and tip JDK version releases.
11+
- The `zulu8`, `zulu11`, `oracle8`, `ibm8`, `semeru8`, `semeru11`, `semeru17`, `graalvm17`, `graalvm21`, and `graalvm25` variants all contain the base JDKs in addition to the specific JDK from their name.
12+
- The `latest` variant contains the base JDKs and all of the specific JDKs above.
1513

1614
## Development
1715

@@ -26,3 +24,25 @@ And then check the built images:
2624
```bash
2725
./build --test
2826
```
27+
28+
## Testing
29+
30+
Images are built per PR for ease in testing. These test images are prefixed with `N_merge-`, where N is the PR number. See the [GitHub Container Registry](https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build) for examples.
31+
32+
To test these images in `dd-trace-java` CI:
33+
34+
1. Open a PR in [DataDog/dd-trace-java-docker-build](https://github.com/DataDog/dd-trace-java-docker-build) with the changes you want to test. It's important that this PR is opened on the latest tip of `master` to ensure that the test images are up-to-date. Let's say these changes are made in PR #123.
35+
2. Run the [generate-test-image-yaml](https://github.com/DataDog/dd-trace-java-docker-build/tree/master/scripts/generate-test-image-yaml.sh) script to generate the YAML snippets that you will need in the following steps:
36+
```bash
37+
./scripts/generate-test-image-yaml.sh 123
38+
```
39+
Note that the test images have `renovate` enabled. This means that any changes to your upstream image (i.e. `DataDog/dd-trace-java-docker-build` PR images) will be auto-updated in `DataDog/images`. The update is done via a bot ([example](https://github.com/DataDog/images/pull/8756)).
40+
41+
3. Open a PR in [DataDog/images](https://github.com/DataDog/images) that adds the generated snippets to the corresponding files. This PR should be automatically merged with the `dd-prapprover` bot. If not, then manually merge the PR. The images should be generated a few minutes after the PR lands on `master`. One way to confirm this is by pulling a test image locally, e.g. by running:
42+
```bash
43+
docker pull registry.ddbuild.io/images/mirror/datadog/dd-trace-java-docker-build:123_merge-base
44+
```
45+
4. Open a PR in [DataDog/dd-trace-java](https://github.com/DataDog/dd-trace-java) that updates the `TESTER_IMAGE_VERSION_PREFIX` variable according to the output from step 2. Here, you can check your test images with `DataDog/dd-trace-java` CI.
46+
5. For each following change made to your original PR #123, ensure the test image (i.e. prefixed with `123_merge-`) SHAs in `DataDog/images` are updated. This should be done via the bot mentioned in step 2 but can also be updated manually.
47+
6. When the test images look good and `DataDog/dd-trace-java` CI is green, merge your `DataDog/dd-trace-java-docker-build` PR #123, close the test `DataDog/dd-trace-java` PR, and **revert the `DataDog/images` PR**.
48+
7. Finally, confirm that the original image (i.e. no prefix) SHAs in `DataDog/images` are updated.

‎scripts/generate-test-image-yaml.sh‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
#!/usr/bin/env bash
2+
# In order to test the images built by your PR in dd-trace-java CI, the images must first be mirrored into the DataDog/images repo.
3+
# Use this script to generate the YAML snippets that need to be added to mirror.yaml and mirror.lock.yaml in DataDog/images.
4+
# Before running, ensure that the expected images are available here: https://github.com/DataDog/dd-trace-java-docker-build/pkgs/container/dd-trace-java-docker-build.
5+
#
6+
# Usage:
7+
# ./scripts/generate-test-image-yaml.sh <PR_NUM>
8+
#
9+
# Example:
10+
# ./scripts/generate-test-image-yaml.sh 138
11+
#
12+
# Requires:
13+
# crane (install with `brew install crane`)
14+
set -euo pipefail
15+
16+
readonly SOURCE_REPO="ghcr.io/datadog/dd-trace-java-docker-build"
17+
18+
readonly CI_VARIANTS=(
19+
base
20+
7
21+
8
22+
11
23+
17
24+
21
25+
25
26+
tip
27+
zulu8
28+
zulu11
29+
oracle8
30+
ibm8
31+
semeru8
32+
semeru11
33+
semeru17
34+
graalvm17
35+
graalvm21
36+
graalvm25
37+
)
38+
39+
if [[ $# -ne 1 ]]; then
40+
echo "Usage: $0 <PR_NUM>" >&2
41+
echo "Example: $0 138" >&2
42+
exit 1
43+
fi
44+
45+
PR_NUM="$1"
46+
if ! [[ "$PR_NUM" =~ ^[0-9]+$ ]]; then
47+
echo "Error: PR_NUM must be a number (got: '$PR_NUM')" >&2
48+
exit 1
49+
fi
50+
51+
if ! command -v crane &>/dev/null; then
52+
echo "Error: 'crane' is required but not found. Install with 'brew install crane'." >&2
53+
exit 1
54+
fi
55+
56+
readonly PREFIX="${PR_NUM}_merge-"
57+
58+
echo "Resolving digests for ${#CI_VARIANTS[@]} variants (prefix: '${PREFIX}')..." >&2
59+
60+
declare -A DIGESTS
61+
for variant in "${CI_VARIANTS[@]}"; do
62+
tag="${PREFIX}${variant}"
63+
echo -n " ${tag} ... " >&2
64+
digest="$(crane digest "${SOURCE_REPO}:${tag}")"
65+
DIGESTS["$variant"]="$digest"
66+
echo "$digest" >&2
67+
done
68+
69+
echo "" >&2
70+
71+
cat <<EOF
72+
### Add to https://github.com/DataDog/images/blob/master/mirror.yaml:
73+
74+
EOF
75+
76+
for variant in "${CI_VARIANTS[@]}"; do
77+
tag="${PREFIX}${variant}"
78+
cat <<EOF
79+
- source: "${SOURCE_REPO}:${tag}"
80+
dest:
81+
repo: "dd-trace-java-docker-build"
82+
tag: "${tag}"
83+
replication_target: ""
84+
renovate:
85+
enabled: true
86+
EOF
87+
done
88+
89+
cat <<EOF
90+
91+
### Add to https://github.com/DataDog/images/blob/master/mirror.lock.yaml:
92+
93+
EOF
94+
95+
for variant in "${CI_VARIANTS[@]}"; do
96+
tag="${PREFIX}${variant}"
97+
cat <<EOF
98+
# renovate
99+
- source: ${SOURCE_REPO}:${tag}
100+
digest: ${DIGESTS[$variant]}
101+
EOF
102+
done
103+
104+
cat <<EOF
105+
106+
### Set in https://github.com/DataDog/dd-trace-java/blob/master/.gitlab-ci.yml:
107+
108+
TESTER_IMAGE_VERSION_PREFIX: "${PREFIX}"
109+
EOF

0 commit comments

Comments
 (0)