Skip to content

Commit dd9a386

Browse files
Fortune-NdlovuFortune-Ndlovu
authored
chore: Use oc-mirror v2 as v1 is deprecated in OCP 4.18 (#1646)
* chore: Use oc-mirror v2 as v1 is deprecated in OCP 4.18 Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * update: remove flags not supported in oc-mirrorv2 '--skip-missing: No equivalent in v2 (images must exist or command fails)' AND '--dest-skip-tls: Replaced with --dest-tls-verify=false' AND '--continue-on-error: No equivalent in v2 (command stops on first error)' Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fix: use existing auth file instead of yq filter that returns null Replace complex yq filter generation that was returning null values with direct use of existing auth file containing valid credentials. Prevents authentication failures in oc-mirror v2 workflow. Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * update: Add --workspace flag to all oc-mirror commands for proper cluster resource generation AND Add --cache-dir option for persistent caching to improve performance and enable incremental mirroring AND Fix ImageSetConfiguration channel structure to ensure proper YAML formatting AND Add cache directory support with default location at /home/fndlovu/.oc-mirror-cache Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fix: resolve oc-mirror v2 authentication and workflow compatibility issues BY Fixing REGISTRY_AUTH_FILE parsing error by using --authfile flag instead of environment variable turns oc-mirror v2 has a bug where it tries to parse REGISTRY_AUTH_FILE path as YAML configuration instead of using it as an auth file path, causing 'StorageDriver not registered' panic. AND I ALSO Fixed disk-to-mirror workflow by adding file:// prefix to --from parameter oc-mirror v2 requires file:// prefix when using --from flag for disk-to-mirror workflow without this prefix, command fails with 'when --from is used, it must have file:// prefix' error AND I Fixe workflow conflict by removing --workspace flag from disk-to-mirror commands oc-mirror v2 doesn't allow --from and --workspace flags together for docker:// destinations --from assumes disk-to-mirror workflow, --workspace assumes mirror-to-mirror workflow - Add proper auth file detection and fallback handling Script now detects existing auth files and uses --authfile flag when available Falls back gracefully when no auth file is present These fixes ensure all three oc-mirror v2 workflows work correctly: - Mirror-to-disk: file://destination (no --workspace needed) - Disk-to-mirror: --from file://source docker://destination (no --workspace) - Mirror-to-mirror: --workspace file://workspace docker://destination Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fix spellcheck Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: no need to remove echo (nice-to-have) AND no need for wc -l < original worked fine. Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: dont treat authfile and cache's entire sting as a single arg Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: provide warning of rosa clusters or clusters hosted as control planes AND remove cache flag users can apply --oc-mirror-flags Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: remove merge_regestry_auth() just use existing auth file directly Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * clean up unset REGISTRY_AUTH_FILE env, as its not needed. Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * update: -c and --config work the same, lets me more clear. Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: add --max-nested-paths=2 as its valid in oc mirror v2 Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: remove empty line Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: usage outline Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * cleanup: no direct --cache-option, users can specify --oc-mirror-flags option Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fixup: remove redundent auth setup as oc-mirror --v2 has a default behaviour to find the auth file at /run/user/4213390/containers/auth.json Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * Improve cluster resources processing logic Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * switch to --config= Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * attempts to create DMS/ITMS resources on HCP clusters, rather than skipping Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * fix wording Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * update: try to apply IDMS/ITMS => If it fails, catch error => Show warning => Continue anyway Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> * DRY-process both IDMS and ITMS file Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com> --------- Signed-off-by: Fortune Ndlovu <fndlovu@redhat.com>
1 parent 163bdb7 commit dd9a386

1 file changed

Lines changed: 87 additions & 96 deletions

File tree

‎.rhdh/scripts/prepare-restricted-environment.sh‎

Lines changed: 87 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
#!/bin/bash
22
#
33
# Script to streamline installing the official RHDH Catalog Source in a disconnected OpenShift or Kubernetes cluster.
4+
# Can be instructed to use oc-mirror v2 for image mirroring (v1 is deprecated in OCP 4.18+).
45
#
5-
# Requires: oc (OCP) or kubectl (K8s), jq, yq, umoci, base64, opm, skopeo
6+
# Requires: oc (OCP) or kubectl (K8s), jq, yq, umoci, base64, opm, skopeo, oc-mirror v2
67

78
set -euo pipefail
89

@@ -99,11 +100,16 @@ Options:
99100
the images to your private registry.
100101
--install-operator <true|false> : Install the RHDH operator right after creating the CatalogSource (default: true)
101102
--extra-images <list> : Comma-separated list of extra images to mirror
102-
--use-oc-mirror <true|false> : Whether to use the 'oc-mirror' tool (default: false).
103+
--use-oc-mirror <true|false> : Whether to use the 'oc-mirror' tool v2 (default: false).
103104
This is the recommended way for mirroring on regular OpenShift clusters.
104-
Bear in mind however that this relies on resources like ImageContentSourcePolicy,
105-
which don't seem to work well on ROSA clusters or clusters with hosted control
105+
oc-mirror v2 generates ImageDigestMirrorSet and ImageTagMirrorSet resources
106+
instead of ImageContentSourcePolicy. Bear in mind however that ImageDigestMirrorSet
107+
and ImageTagMirrorSet don't seem to work well on ROSA clusters or clusters with hosted control
106108
planes (like HyperShift or Red Hat OpenShift on IBM Cloud).
109+
IMPORTANT: When using --to-dir (mirrorToDisk workflow), oc-mirror v2 only copies images
110+
to disk and does NOT create cluster resources (CatalogSource, IDMS, ITMS). These resources
111+
are only generated when pushing to a registry (mirrorToMirror workflow or diskToMirror workflow) using --from-dir and
112+
--to-registry together.
107113
--oc-mirror-path <path> : Path to the oc-mirror binary (default: 'oc-mirror').
108114
--oc-mirror-flags <string> : Additional flags to pass to all oc-mirror commands.
109115
--install-yq : Install yq $YQ_VERSION from https://github.com/mikefarah/yq (not the jq python wrapper)
@@ -134,6 +140,18 @@ Examples:
134140
$0 \\
135141
--ci-index true \\
136142
--filter-versions '1.4,1.5'
143+
144+
# WORKFLOW with oc-mirror v2 for fully disconnected environments:
145+
# (on connected host): Export images to disk
146+
$0 \\
147+
--use-oc-mirror true \\
148+
--to-dir /path/to/export
149+
150+
# (on disconnected bastion): Push images to registry and create cluster resources
151+
$0 \\
152+
--use-oc-mirror true \\
153+
--from-dir /path/to/export \\
154+
--to-registry registry.example.com
137155
"
138156
}
139157

@@ -381,50 +399,6 @@ else
381399
YQ=$(which yq)
382400
fi
383401

384-
function merge_registry_auth() {
385-
set -euo pipefail
386-
387-
currentRegistryAuthFile="${REGISTRY_AUTH_FILE:-${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/containers/auth.json}"
388-
debugf "currentRegistryAuthFile: $currentRegistryAuthFile"
389-
if [ ! -f "${currentRegistryAuthFile}" ]; then
390-
debugf "Missing registry auth file. Will proceed without any existing auth against the registry hosting the index image: $INDEX_IMAGE"
391-
return
392-
fi
393-
# TODO(rm3l): Overriding XDG_RUNTIME_DIR so it can work with "oc-mirror v1", which does not work with REGISTRY_AUTH_FILE.
394-
# Remove this when oc-mirror v2 is out of TP.
395-
XDG_RUNTIME_DIR=$(mktemp -d)
396-
export XDG_RUNTIME_DIR
397-
mkdir -p "${XDG_RUNTIME_DIR}/containers"
398-
# shellcheck disable=SC2064
399-
trap "rm -fr $XDG_RUNTIME_DIR || true" EXIT
400-
# Using the current working dir, otherwise tools like 'skopeo login' will attempt to write to /run, which
401-
# might be restricted in CI environments.
402-
# This also ensures that the credentials don't conflict with any existing creds for the same registry
403-
export REGISTRY_AUTH_FILE="${XDG_RUNTIME_DIR}/containers/auth.json"
404-
debugf "REGISTRY_AUTH_FILE: $REGISTRY_AUTH_FILE"
405-
406-
# Merge existing authentication from currentRegistryAuthFile into REGISTRY_AUTH_FILE
407-
images=("${INDEX_IMAGE}")
408-
if [[ -n "${TO_REGISTRY}" ]]; then
409-
images+=("$(buildRegistryUrl)")
410-
fi
411-
registries=("registry.redhat.io" "quay.io")
412-
for img in "${images[@]}"; do
413-
reg=$(echo "$img" | cut -d'/' -f1)
414-
[[ " ${registries[*]} " =~ " $reg " ]] || registries+=("$reg")
415-
done
416-
tmpFile=$(mktemp)
417-
# shellcheck disable=SC2064
418-
trap "rm -f $tmpFile || true" EXIT
419-
echo '{"auths": {' >"$tmpFile"
420-
for reg in "${registries[@]}"; do
421-
echo " \"$reg\": .auths.\"$reg\"," >>"$tmpFile"
422-
done
423-
sed -i '$ s/,$//' "$tmpFile"
424-
echo '}}' >>"$tmpFile"
425-
debugf "yq filter: $(cat "$tmpFile")"
426-
"$YQ" -o=json "$(cat "$tmpFile")" "${currentRegistryAuthFile}" >"${REGISTRY_AUTH_FILE}"
427-
}
428402

429403
function ocp_prepare_internal_registry() {
430404
set -euo pipefail
@@ -901,35 +875,23 @@ if [[ "${IS_OPENSHIFT}" = "true" && "${TO_REGISTRY}" = "OCP_INTERNAL" ]]; then
901875
ocp_prepare_internal_registry
902876
fi
903877

904-
merge_registry_auth
905878

906879
manifestsTargetDir="${TMPDIR}"
907880
if [[ -n "${FROM_DIR}" ]]; then
908881
manifestsTargetDir="${FROM_DIR}"
909882
fi
910883

911884
if [[ "${USE_OC_MIRROR}" = "true" ]]; then
912-
# TODO(rm3l): oc-mirror v1 always loads the docker creds first:
913-
# https://github.com/openshift/oc-mirror/blob/main/pkg/image/credentials.go
914-
# But we want to use our own credentials file, which is not possible until oc-mirror v2 (currently tech preview)
915-
if [ -f ~/.docker/config.json ]; then
916-
debugf "Temporarily moving ~/.docker/config.json to ~/.docker/config.json.bak, so as to work with oc-mirror v1"
917-
mv -f ~/.docker/config.json ~/.docker/config.json.bak || true
918-
trap "mv -f ~/.docker/config.json.bak ~/.docker/config.json || true" EXIT
919-
fi
885+
# oc-mirror v2 uses ${XDG_RUNTIME_DIR}/containers/auth.json by default for authentication
920886

921887
NAMESPACE_CATALOGSOURCE="openshift-marketplace"
922888
ocMirrorLogFile="${TMPDIR}/oc-mirror.log.txt"
889+
923890
if [[ -z "${FROM_DIR}" ]]; then
924891
# Direct to registry
925892
cat <<EOF >"${TMPDIR}/imageset-config.yaml"
926-
apiVersion: mirror.openshift.io/v1alpha2
893+
apiVersion: mirror.openshift.io/v2alpha1
927894
kind: ImageSetConfiguration
928-
storageConfig:
929-
local:
930-
# Do not delete or modify metadata generated by the oc-mirror plugin,
931-
# use the same storage backend every time run the oc-mirror plugin for the same mirror
932-
path: ./metadata
933895
mirror:
934896
operators:
935897
- catalog: ${INDEX_IMAGE}
@@ -948,7 +910,6 @@ EOF
948910
- name: fast-${v}
949911
EOF
950912
done
951-
952913
fi
953914
nbExtraImgs=${#EXTRA_IMAGES[@]}
954915
if [ "$nbExtraImgs" -ge 1 ]; then
@@ -966,11 +927,10 @@ EOF
966927
"${OC_MIRROR_PATH}" \
967928
--config="${TMPDIR}/imageset-config.yaml" \
968929
file://"${TO_DIR}" \
969-
--skip-missing \
970-
--dest-skip-tls \
971-
--continue-on-error \
972-
--max-nested-paths=1 \
973-
"$OC_MIRROR_FLAGS" |
930+
--dest-tls-verify=false \
931+
--max-nested-paths=2 \
932+
"$OC_MIRROR_FLAGS" \
933+
--v2 |
974934
tee "${ocMirrorLogFile}"
975935
if [[ "${TO_DIR}" != "${TMPDIR}" ]]; then
976936
cp -f "${TMPDIR}/imageset-config.yaml" "${TO_DIR}/imageset-config.yaml"
@@ -993,12 +953,12 @@ EOF
993953

994954
"${OC_MIRROR_PATH}" \
995955
--config="${TMPDIR}/imageset-config.yaml" \
956+
--workspace file://"${TMPDIR}" \
996957
"docker://${registryUrl}" \
997-
--skip-missing \
998-
--dest-skip-tls \
999-
--continue-on-error \
958+
--dest-tls-verify=false \
1000959
--max-nested-paths=2 \
1001-
"$OC_MIRROR_FLAGS" |
960+
"$OC_MIRROR_FLAGS" \
961+
--v2 |
1002962
tee "${ocMirrorLogFile}"
1003963
fi
1004964

@@ -1025,36 +985,67 @@ EOF
1025985

1026986
"${OC_MIRROR_PATH}" \
1027987
--config="${FROM_DIR}/imageset-config.yaml" \
1028-
--from "${FROM_DIR}" \
988+
--from file://"${FROM_DIR}" \
1029989
"docker://${registryUrl}" \
1030-
--skip-missing \
1031-
--dest-skip-tls \
1032-
--continue-on-error \
1033-
"$OC_MIRROR_FLAGS" |
990+
--dest-tls-verify=false \
991+
--max-nested-paths=2 \
992+
"$OC_MIRROR_FLAGS" \
993+
--v2 |
1034994
tee "${ocMirrorLogFile}"
1035995
fi
1036996
fi
1037997

1038-
if [ -f "${ocMirrorLogFile}" ]; then
1039-
# The xargs here is to trim whitespaces
1040-
catalogSourceLocation=$(sed -n -e 's/^Writing CatalogSource manifests to \(.*\)$/\1/p' "${ocMirrorLogFile}" | xargs)
1041-
icspLocation=$(sed -n -e 's/^Writing ICSP manifests to \(.*\)$/\1/p' "${ocMirrorLogFile}" | xargs)
1042-
if [[ -n "${icspLocation}" ]]; then
1043-
debugf "ICSP parent location: ${TMPDIR}/${icspLocation}"
1044-
if [[ -n "${TO_REGISTRY}" ]]; then
1045-
invoke_cluster_cli apply -f "${TMPDIR}"/"${icspLocation}"/imageContentSourcePolicy.yaml
1046-
fi
998+
# oc-mirror v2 generates manifests in working-dir/cluster-resources/
999+
# These are only generated during diskToMirror operations (when TO_REGISTRY is set)
1000+
if [[ -n "${TO_REGISTRY}" ]]; then
1001+
clusterResourcesDir="${TMPDIR}/working-dir/cluster-resources"
1002+
if [[ -n "${FROM_DIR}" ]]; then
1003+
clusterResourcesDir="${FROM_DIR}/working-dir/cluster-resources"
10471004
fi
1048-
if [[ -n "${catalogSourceLocation}" ]]; then
1049-
# Replace some metadata and add the default list of secrets
1050-
debugf "catalogSource parent location: ${TMPDIR}/${catalogSourceLocation}"
1051-
"$YQ" -i '.metadata.name = "rhdh-catalog"' -i "${TMPDIR}"/"${catalogSourceLocation}"/catalogSource-*.yaml
1052-
"$YQ" -i '.spec.displayName = "Red Hat Developer Hub Catalog (Airgapped)"' -i "${TMPDIR}"/"${catalogSourceLocation}"/catalogSource-*.yaml
1053-
"$YQ" -i '.spec.secrets = (.spec.secrets // []) + ["internal-reg-auth-for-rhdh", "internal-reg-ext-auth-for-rhdh", "reg-pull-secret"]' \
1054-
"${TMPDIR}"/"${catalogSourceLocation}"/catalogSource-*.yaml
1055-
if [[ -n "${TO_REGISTRY}" ]]; then
1056-
invoke_cluster_cli apply -f "${TMPDIR}"/"${catalogSourceLocation}"/catalogSource-*.yaml
1005+
1006+
if [ -d "${clusterResourcesDir}" ]; then
1007+
debugf "Processing cluster resources from: ${clusterResourcesDir}"
1008+
1009+
# Apply ImageDigestMirrorSet and ImageTagMirrorSet resources
1010+
foundResources=false
1011+
# oc-mirror v2 generates files with patterns: idms-*.yaml, itms-*.yaml
1012+
1013+
for manifest in "${clusterResourcesDir}"/idms-*.yaml "${clusterResourcesDir}"/imageDigestMirrorSet*.yaml \
1014+
"${clusterResourcesDir}"/itms-*.yaml "${clusterResourcesDir}"/imageTagMirrorSet*.yaml; do
1015+
if [[ -f "${manifest}" ]]; then
1016+
debugf "Applying manifest: ${manifest}"
1017+
if ! invoke_cluster_cli apply -f "${manifest}" 2>&1; then
1018+
warnf "Failed to apply manifest: ${manifest}"
1019+
if [[ "${IS_HOSTED_CONTROL_PLANE}" = "true" ]]; then
1020+
warnf "This is expected on Hosted Control Plane clusters (ROSA HCP, HyperShift, etc.)."
1021+
warnf "On HCP clusters, image mirrors must be configured in the HostedCluster resource on the management cluster."
1022+
warnf "Please work with your cluster administrator to configure image mirroring."
1023+
fi
1024+
fi
1025+
foundResources=true
1026+
fi
1027+
done
1028+
1029+
# Process CatalogSource resources
1030+
# oc-mirror v2 generates files with pattern: cs-*.yaml
1031+
for manifest in "${clusterResourcesDir}"/cs-*.yaml "${clusterResourcesDir}"/catalogSource*.yaml; do
1032+
if [[ -f "${manifest}" ]]; then
1033+
debugf "Processing CatalogSource: ${manifest}"
1034+
# Replace some metadata and add the default list of secrets
1035+
"$YQ" -i '.metadata.name = "rhdh-catalog"' "${manifest}"
1036+
"$YQ" -i '.spec.displayName = "Red Hat Developer Hub Catalog (Airgapped)"' "${manifest}"
1037+
"$YQ" -i '.spec.secrets = (.spec.secrets // []) + ["internal-reg-auth-for-rhdh", "internal-reg-ext-auth-for-rhdh"]' "${manifest}"
1038+
"$YQ" -i '.spec.image |= sub("default-route-openshift-image-registry\.apps\.[^/]+", "image-registry.openshift-image-registry.svc:5000")' "${manifest}"
1039+
invoke_cluster_cli apply -f "${manifest}"
1040+
foundResources=true
1041+
fi
1042+
done
1043+
1044+
if [[ "${foundResources}" != "true" ]]; then
1045+
warnf "No cluster resources found in ${clusterResourcesDir}. This may indicate that oc-mirror v2 did not generate them."
10571046
fi
1047+
else
1048+
warnf "Cluster resources directory not found: ${clusterResourcesDir}. This may indicate that oc-mirror v2 did not generate cluster resources."
10581049
fi
10591050
fi
10601051
else

0 commit comments

Comments
 (0)