Skip to content

OCPBUGS-92206: bump catalog image version cap from 4.21 to 4.22#8836

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
muraee:assistant/task-206
Jun 29, 2026
Merged

OCPBUGS-92206: bump catalog image version cap from 4.21 to 4.22#8836
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
muraee:assistant/task-206

Conversation

@muraee

@muraee muraee commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Updates the catalog image version cap from 4.21 to 4.22 now that 4.22 catalog images are available.

Related Issue

OCPBUGS-92206

Testing

Co-Authored-By: Claude Haiku 4.5 noreply@anthropic.com

Summary by CodeRabbit

  • Bug Fixes
    • Updated catalog image selection to cap lookback at the latest supported OpenShift catalog release (4.22), improving which “latest known-good” catalog images are chosen for newer versions.
    • Adjusted and expanded related tests to match the updated selection rules and fallback behavior.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 25, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@muraee: This pull request references Jira Issue OCPBUGS-92206, which is invalid:

  • expected the bug to target either version "5.0." or "openshift-5.0.", but it targets "4.21.z" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

Updates the catalog image version cap from 4.21 to 4.22 now that 4.22 catalog images are available.

Related Issue

OCPBUGS-92206

Testing

Co-Authored-By: Claude Haiku 4.5 noreply@anthropic.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Jun 25, 2026
@openshift-ci openshift-ci Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/needs-area labels Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@muraee muraee marked this pull request as ready for review June 25, 2026 16:03
@muraee

muraee commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 25, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@muraee: This pull request references Jira Issue OCPBUGS-92206, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (wewang@redhat.com), skipping review request.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2026
@muraee

muraee commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/area control-plane-operator

@openshift-ci openshift-ci Bot added the area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release label Jun 25, 2026
@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: e864dc19-4e52-48f6-9558-9ee3ff15ce47

📥 Commits

Reviewing files that changed from the base of the PR and between cbf7d1c and ad7b62a.

📒 Files selected for processing (2)
  • support/catalogs/images.go
  • support/catalogs/images_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • support/catalogs/images.go

📝 Walkthrough

Walkthrough

support/catalogs/images.go updates computeCatalogImages to cap catalogVersion at OpenShift catalog semver 4.22 instead of 4.21. support/catalogs/images_test.go updates the expected catalog image selection for 5.0.0, 5.0.0-0.nightly-multi-*, fallback behavior when 4.22 images are unavailable, and the 4.22.0 case.

🚥 Pre-merge checks | ✅ 11
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: raising the catalog image version cap from 4.21 to 4.22.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The changed tests use static subtest names; no Ginkgo titles or variable-built names include dates, UUIDs, node names, or other dynamic data.
Test Structure And Quality ✅ Passed The added tests are deterministic table-driven unit tests with no cluster setup, cleanup, or waits, and their assertion style matches existing repo patterns.
Topology-Aware Scheduling Compatibility ✅ Passed The PR only changes catalog image version cap logic and tests; no manifests, controllers, replicas, affinities, selectors, or topology-aware scheduling were introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; changes are unit tests in support/catalogs with mocked image lookups and no IPv4 or external connectivity assumptions.
No-Weak-Crypto ✅ Passed Patch only raises the catalog version cap and updates tests; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons appear in changed lines.
Container-Privileges ✅ Passed PASS: The PR only changes Go source/tests in support/catalogs, and those files contain no privileged/K8s manifest settings like hostPID, privileged, or allowPrivilegeEscalation.
No-Sensitive-Data-In-Logs ✅ Passed PASS: The PR only changes the catalog version cap and tests; no logging APIs or log statements were added, and no sensitive data appears in the changed code.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci-robot

Copy link
Copy Markdown

@muraee: This pull request references Jira Issue OCPBUGS-92206, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (wewang@redhat.com), skipping review request.

Details

In response to this:

Summary

Updates the catalog image version cap from 4.21 to 4.22 now that 4.22 catalog images are available.

Related Issue

OCPBUGS-92206

Testing

Co-Authored-By: Claude Haiku 4.5 noreply@anthropic.com

Summary by CodeRabbit

  • Bug Fixes
  • Updated catalog image version handling to recognize the latest supported OpenShift catalog release, improving catalog image selection for newer versions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release label Jun 25, 2026
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: muraee

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 25, 2026
@openshift-ci openshift-ci Bot requested review from Nirshal and devguyio June 25, 2026 16:08

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@support/catalogs/images.go`:
- Line 172: The cap behavior changed in images.go by updating maxCatalogVersion,
so the downstream expectation in images_test.go is now outdated. Update the test
case that currently maps 4.22.0 to v4.21 so it expects v4.22 instead, keeping
the CatalogForVersion cap contract aligned with the new ceiling.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 25c2b70e-f8d7-4209-b14b-a27607603029

📥 Commits

Reviewing files that changed from the base of the PR and between 02675db and cbf7d1c.

📒 Files selected for processing (1)
  • support/catalogs/images.go

Comment thread support/catalogs/images.go
@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 42.18%. Comparing base (6a6d8c1) to head (ad7b62a).
⚠️ Report is 390 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8836      +/-   ##
==========================================
+ Coverage   40.68%   42.18%   +1.50%     
==========================================
  Files         755      769      +14     
  Lines       93368    96977    +3609     
==========================================
+ Hits        37985    40909    +2924     
- Misses      52649    53220     +571     
- Partials     2734     2848     +114     
Files with missing lines Coverage Δ
support/catalogs/images.go 78.72% <100.00%> (ø)

... and 135 files with indirect coverage changes

Flag Coverage Δ
cmd-support 35.63% <100.00%> (+0.92%) ⬆️
cpo-hostedcontrolplane 45.27% <ø> (+3.46%) ⬆️
cpo-other 45.10% <ø> (+3.70%) ⬆️
hypershift-operator 50.40% <ø> (-0.42%) ⬇️
other 31.69% <ø> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@muraee

muraee commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

/address-review-comments

@github-actions

Copy link
Copy Markdown

🤖 Addressing review comments: workflow run

The catalog image version was capped at 4.21 while waiting for 4.22
catalog images to be available. Now that 4.22 catalog images are
ready, update the cap to 4.22.

Signed-off-by: Mulham Raee <mulham.raee@redhat.com>
Commit-Message-Assisted-by: Claude (via Claude Code)
@github-actions github-actions Bot force-pushed the assistant/task-206 branch from cbf7d1c to ad7b62a Compare June 26, 2026 09:44
@csrwng

csrwng commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 26, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@muraee

muraee commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

/verified by unit

@muraee

muraee commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

/cherry-pick release-4.22

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jun 26, 2026
@openshift-cherrypick-robot

Copy link
Copy Markdown

@muraee: once the present PR merges, I will cherry-pick it on top of release-4.22 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-4.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot

Copy link
Copy Markdown

@muraee: This PR has been marked as verified by unit.

Details

In response to this:

/verified by unit

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aks | Build: 2070505370224168960 | Cost: $3.6616934999999997 | Failed step: hypershift-azure-run-e2e

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@cwbotbot

cwbotbot commented Jun 26, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

e2e-aks

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD c180b15 and 2 for PR HEAD ad7b62a in total

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aks | Build: 2070610729685225472 | Cost: $1.9644927500000007 | Failed step: hypershift-azure-run-e2e

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@muraee

muraee commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

/retest-required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 2d2b2d0 and 1 for PR HEAD ad7b62a in total

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aks | Build: 2071514455455305728 | Cost: $2.9446942500000004 | Failed step: hypershift-azure-run-e2e

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

Now I have all the evidence I need. Here is the final report:

Test Failure Analysis Complete

Job Information

Test Failure Analysis

Error

Container packageserver in pod packageserver-86ddf864b7-9fn9g has a restartCount > 0 (1)
Container packageserver in pod packageserver-86ddf864b7-rtdcl has a restartCount > 0 (1)

Summary

The TestCreateClusterHABreakGlassCredentials test failed because its EnsureNoCrashingPods sub-check detected that two packageserver pods in the hosted cluster had each restarted once. Both pods crashed at startup (~09:45:23-24Z) with level=fatal msg="failed to apply cluster TLS profile to serving options: failed to get APIServer config: apiservers.config.openshift.io \"cluster\" not found", then recovered successfully within 1 second. This is a pre-existing OLM startup race condition — the packageserver starts before the apiservers.config.openshift.io/cluster resource exists in the hosted cluster — and is completely unrelated to PR #8836's changes (bumping maxCatalogVersion from 4.21 to 4.22 in support/catalogs/images.go). All 7 other tests that run EnsureNoCrashingPods passed; only this one hosted cluster instance hit the timing window.

Root Cause

The root cause is a pre-existing startup race condition in the OLM packageserver within HyperShift hosted clusters:

  1. During hosted control plane initialization for cluster ha-break-glass-creds-gx2bg, three packageserver pods were created via deployment packageserver-86ddf864b7.
  2. Two of the three pods (-9fn9g and -rtdcl) started before the apiservers.config.openshift.io/cluster APIServer config resource was available in the hosted cluster's apiserver.
  3. The packageserver startup code attempted to read the TLS profile from this resource (failed to get APIServer config: apiservers.config.openshift.io "cluster" not found) and exited with level=fatal, exit code 1.
  4. Both pods restarted and succeeded on the second attempt (the APIServer config resource became available within ~1 second). Both pods were Running and Ready at test validation time.
  5. However, the EnsureNoCrashingPods test sub-check has zero tolerance for restartCount > 0, so it flagged these restart-recovered pods as failures.

The third packageserver pod (-n98wp) did not crash — likely started slightly later, after the resource was available.

This failure is NOT caused by PR #8836. The PR only modifies support/catalogs/images.go (bumping maxCatalogVersion) and its test file. These changes have no connection to OLM packageserver startup, TLS profile configuration, or APIServer config resource availability.

Classification: Flaky / pre-existing infrastructure race condition.

Recommendations
  1. Retry the CI job — this is a transient race condition unlikely to reproduce consistently. The PR changes are safe.
  2. File a bug against OLM to add retry/backoff logic in packageserver when reading apiservers.config.openshift.io/cluster during startup, rather than exiting fatally on the first attempt.
  3. Consider adjusting EnsureNoCrashingPods (in util.go:829) to tolerate restartCount == 1 during initial hosted cluster bringup, as brief startup races are expected in HyperShift's hosted control plane lifecycle.
Evidence
Evidence Detail
Failed test TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster/EnsureNoCrashingPods (0.30s)
Failing pod 1 packageserver-86ddf864b7-9fn9g — restartCount=1, exitCode=1, crashed at 09:45:24Z
Failing pod 2 packageserver-86ddf864b7-rtdcl — restartCount=1, exitCode=1, crashed at 09:45:23Z
Fatal error failed to apply cluster TLS profile to serving options: failed to get APIServer config: apiservers.config.openshift.io "cluster" not found
Recovery Both pods restarted and were Running+Ready by 09:45:25Z (~1s recovery)
Other tests 7 other tests ran EnsureNoCrashingPods successfully — only this hosted cluster hit the timing window
PR #8836 changes Only support/catalogs/images.go (maxCatalogVersion 4.21→4.22) and support/catalogs/images_test.go — unrelated to OLM/packageserver
Test results 396 total, 347 passed, 46 skipped, 3 failures (all from same test hierarchy)
Failed CI step hypershift-azure-run-e2e (test phase) — all pre/post steps passed

@muraee

muraee commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

/retest-required

@openshift-ci

openshift-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

@muraee: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot Bot merged commit e680e1d into openshift:main Jun 29, 2026
41 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@muraee: Jira Issue Verification Checks: Jira Issue OCPBUGS-92206
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-92206 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

Summary

Updates the catalog image version cap from 4.21 to 4.22 now that 4.22 catalog images are available.

Related Issue

OCPBUGS-92206

Testing

Co-Authored-By: Claude Haiku 4.5 noreply@anthropic.com

Summary by CodeRabbit

  • Bug Fixes
  • Updated catalog image selection to cap lookback at the latest supported OpenShift catalog release (4.22), improving which “latest known-good” catalog images are chosen for newer versions.
  • Adjusted and expanded related tests to match the updated selection rules and fallback behavior.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-cherrypick-robot

Copy link
Copy Markdown

@muraee: new pull request created: #8861

Details

In response to this:

/cherry-pick release-4.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-robot

Copy link
Copy Markdown
Contributor

Fix included in release 5.0.0-0.nightly-2026-06-30-215228

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants