Conversation
I reviewed the pipeline tasks to figure out if optional tasks can be skipped using native "when" conditions. In some cases this is applicable, however there are still limitation where it can't be used. To document a guidelines I am adding a doc with examples and decision tree of where the "when" can or can't be used. Signed-off-by: Ales Raszka <araszka@redhat.com>
A subset of tasks in the hosted pipeline can run in parallel to speedup the pipelinerun execution time. I identified several tasks and updated pipeline dependency graph. There are more tasks that could be also parallelized but it makes the graph readability more difficult. This solution is compromise between speed and maintainability. Signed-off-by: Ales Raszka <araszka@redhat.com>
Review Summary by QodoParallelize pipeline tasks and add when condition guidelines
WalkthroughsDescription• Parallelize pipeline tasks by updating dependency graph - Multiple tasks now depend on detect-changes instead of sequential chains - Reduces overall pipeline execution time while maintaining readability • Add when conditions to conditionally skip tasks - Introduces reusable condition anchors for bundle path and cert project checks - Adds conditions to merge-registry-credentials, digest-pinning, and other tasks • Reorganize task execution order for better parallelization - Move static-tests earlier in pipeline (after apply-test-waivers) - Remove resolve-pr-type and verify-pinned-digest tasks - Consolidate dependencies into get-pyxis-certification-data task • Add documentation guidelines for Tekton when conditions - Document rules for safe usage of when conditions in pipelines - Provide decision tree and examples for when conditions can/cannot be used • Enhance digest-pinning task with enforcement flag - Add enforce_pinning parameter to fail task on pinning violations Diagramflowchart LR
DC["detect-changes"]
DC --> YL["yaml-lint"]
DC --> CP["check-permissions"]
DC --> SGPT["set-github-pr-title"]
DC --> RC["read-config"]
DC --> ATW["apply-test-waivers"]
DC --> CH["content-hash"]
DC --> CPC["certification-project-check"]
DC --> GO["get-organization"]
ATW --> ST["static-tests"]
CPC --> GPCD["get-pyxis-certification-data"]
RC --> GPCD
YL --> GPCD
CP --> GPCD
SGPT --> GPCD
ST --> GPCD
CH --> GPCD
GPCD --> VCF["validate-catalog-format"]
GPCD --> VP["verify-project"]
GO --> VP
VP --> MRC["merge-registry-credentials"]
MRC --> DP["digest-pinning"]
DP --> DFC["dockerfile-creation"]
DFC --> BB["build-bundle"]
File Changes1. ansible/roles/operator-pipeline/templates/openshift/pipelines/operator-hosted-pipeline.yml
|
Code Review by Qodo
1.
|
Signed-off-by: Ales Raszka <araszka@redhat.com>
mantomas
left a comment
There was a problem hiding this comment.
Makes sense to me, one small nitpick.
| workspace: results | ||
| subPath: summary | ||
|
|
||
| - name: resolve-pr-type |
There was a problem hiding this comment.
I am missing the context of the removal of this - was this completely unused?
There was a problem hiding this comment.
Yes it was not used in the hosted pipeline. We probably replaced it some time ago and didn't remove it.
| subPath: base | ||
|
|
||
| - name: apply-test-waivers | ||
| - name: static-tests |
There was a problem hiding this comment.
Does it make a sense to skip static tests with when: *bundleAdded?
There was a problem hiding this comment.
No, the static test also tests catalogs, and they are required in all workflows.
|
|
||
| --- | ||
|
|
||
| ## Rule 4 — `when` input is a task result, guarded task also consumes task results in params ❌ |
There was a problem hiding this comment.
Well this might be safe if source task always run right? Or am I missing something?
There was a problem hiding this comment.
That's the tricky part that made me spend a lot of time trying to debug it. If a task consumes any results, it doesn't matter if the task results in the when section is available or not. I updated text in this section to better explain it. The overall decision tree in this document should help with deciding when it is safe or not safe to use when.
I reviewed the pipeline tasks to figure out if optional tasks can be
skipped using native "when" conditions. In some cases this is
applicable, however there are still limitation where it can't be used.
To document a guidelines I am adding a doc with examples and decision
tree of where the "when" can or can't be used.
The other part of the PR is focused on paralelization:
A subset of tasks in the hosted pipeline can run in parallel to speedup
the pipelinerun execution time. I identified several tasks and updated
pipeline dependency graph. There are more tasks that could be also
parallelized but it makes the graph readability more difficult. This
solution is compromise between speed and maintainability.
Merge Request Checklists