feat(ISV-6697): Release pipeline uses pending index repositories#923
feat(ISV-6697): Release pipeline uses pending index repositories#923JakubDurkac merged 13 commits intoISV-6696-6697from
Conversation
Review Summary by QodoRelease pipeline uses pending index repositories with IIB overwrite support
WalkthroughsDescription• Add IIB overwrite token support to enable pending repository overwrites • Implement build tags suffix for consistent tagging across IIB and publish tasks • Update release pipeline to use pending repositories as index image sources • Remove environment-specific conditionals (dev/qa/preprod) from pipeline tasks • Add new generate-build-timestamp task for Unix timestamp generation • Update Ansible tasks to pass credentials via iib-credentials workspace Diagramflowchart LR
A["Release Pipeline"] -->|"uses pending repos"| B["IIB API"]
B -->|"overwrite token"| C["Pending Repositories"]
D["generate-build-timestamp"] -->|"timestamp suffix"| E["IIB Builds"]
E -->|"same tags"| F["publish-to-index"]
G["iib-credentials workspace"] -->|"username:password"| E
File Changes1. operatorcert/entrypoints/add_fbc_fragments_to_index.py
|
Code Review by Qodo
1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| if [[ "$(workspaces.iib-credentials.bound)" == "true" ]]; then | ||
| IIB_QUAY_USER=$(cat $(workspaces.iib-credentials.path)/username) | ||
| IIB_QUAY_TOKEN=$(cat $(workspaces.iib-credentials.path)/password) | ||
| EXTRA_ARGS+=" --iib-overwrite-token ${IIB_QUAY_USER}:${IIB_QUAY_TOKEN}" |
There was a problem hiding this comment.
We should avoid using cli params to pass secrets. I think better solution is to use env vars and load it from the env vars in the script.
There was a problem hiding this comment.
Fixed in the latest commit. Let me know if you meant it differently.
| IIB module for building a index images for a bundle | ||
| """ | ||
|
|
||
| # pylint: disable=duplicate-code |
There was a problem hiding this comment.
Can this be set only on affected parts of code rather than on the whole file?
There was a problem hiding this comment.
Should be better now.
| # DO NOT USE set -x to avoid revealing credentials in logs! | ||
| set -e | ||
|
|
||
| ENV=$(params.environment) |
There was a problem hiding this comment.
This can be removed, since it is unused after removal of the if condition.
There was a problem hiding this comment.
I am not seeing the removal of the condition in diff anymore. Was it something form different PR that is already present in the target branch?
There was a problem hiding this comment.
I see, thanks for explaining 👍
|
As it seems it will take some time for the overwrite token bug to be fixed by IIB, I've made a simple temporary workaround that we can remove later, doing the overwriting manually. Made it just to see if the tests results look otherwise correct. Here are some tests with logs, you can look if you're interested. I was mainly checking if the sources and destinations are as expected for both add-bundle task, and then the publish task, including the version + permanent tags. Those look correct for these tests:
The 'make build-and-test-isv-fbc-catalog' is currently getting stuck for me on hosted pipeline, I'm trying to find out why. I'll add results if I get some (EDIT: fixed, see comment below). Just FYI @Allda, @RichardPlesnik. Also, please let me know if you would like to run some additional tests, so when the IIB is fixed, we are prepared to move forward. |
|
I've found out that the pipeline was getting stuck on fbc catalog test, because get-supported-versions task was generating results that were too big. Apparently by my addition of max_supported_pending_repository we barely hit over 4 KB limit by a few bytes, which lead to the results being truncated. The "message" attribute of "supported-version-check" step, which is a string holding the JSON result object was missing few versions of the last attribute "indices_ocp_versions" and proper closing brackets, which lead to pipeline getting stuck. We would've probably hit it also if there was like 1 more supported OCP version in the list, so to be completely safe long term, I've removed currently unused repositories_with_version and public_repository_mirrors_with_version from results, which should make the results like 60 % smaller. The last thing we need to pass all tests is for IIB to put a new functioning default reading token to stage, because there's a wrong one active at the moment, therefore pipeline is failing with unauthorized, could not find robot with username... logs: https://gist.github.com/rh-operator-bundle-bot/bcae4a6a66a0ac9a0126374839965694 |
JakubDurkac
force-pushed
the
ISV-6696-6697
branch
from
393f70a to
a8a7001
Compare
JakubDurkac
force-pushed
the
ISV-6696-6697
branch
from
a8a7001 to
68fb548
Compare
openshift-ci
Bot
added
the
needs-rebase
openshift-ci
Bot
removed
the
needs-rebase
| value: "$(params.pipeline_image)" | ||
| - name: index_images | ||
| value: "$(tasks.get-supported-versions.results.public_repository_mirrors_with_version)" | ||
| value: "$(tasks.get-supported-versions.results.max_supported_pending_repository)" |
There was a problem hiding this comment.
Shouldn't this be pending_repositories_with_version?
Allda
left a comment
Allda
left a comment
There was a problem hiding this comment.
I noticed in the pipeline logs that by changing the IIB build source you accidentally change the signing step. The old solution send a signing request with following payload:
"docker_reference": "registry.redhat.io/redhat/community-operator-index:v4.14", "image_name": "redhat/community-operator-index",
The new solution is using a different reference:
'docker_reference': 'quay.io/redhat-pending/redhat----community-operator-index-pending:v4.20', 'errors': [], 'image_name': 'redhat-pending/redhat----community-operator-index-pending',
This needs to be fixed; otherwise, the indexes won't be properly signed.
|
I made changes to sign-index-image task:
When the certificate renewal is merged to main #933, I'll rebase and run the tests. |
JakubDurkac
force-pushed
the
ISV-6696-6697
branch
from
68fb548 to
7339d2a
Compare
1. For release pipeline, use pending repositories as source for index images. 2. Use IIB overwrite token in release pipeline to overwrite the source pending repositories. 3. Specify tags (version + permanent tag) for IIB calls and publish task. 4. Passing of tokens based on publish-to-index task that already uses the token. Added 'set -e' and switched to not using '--verbose' for add-bundle-to-index and build-fbc-index-image, the tasks token is newly passed to.
|
The signing step should be fixed now. Let me know what you think. @Allda |
3 participants
Merging to non-primary branch, so that we can test and merge both this (ISV-6696) and release pipeline changes (ISV-6697) to main at the same time.
WARNING: Currently it seems that the overwrite token passed to IIB API build request is not working due to some bug, therefore the builds fail, because they fail to overwrite "-pending" repositories (unauthorized), so we can't fully test the changes currently. More info here in the ticket I've opened.
Merge Request Checklists