Fix release process

[fabianburth]

Description
The release process failed b/c the conformance tests used the cli build from the main branch. The cli built a component constructor with

input:
  type: file

This is translated to

access:
  type: LocalBlob/v1 # newly introduced capital case

The capital case is only introduced in main and was thus not supported by the controllers.

Decision:

• we need to build and test the controllers together in a workflow, so that they can be tested in the same conformance test run
• we need to build on commits to the release branches, so the cli workflow can use the controllers already built for the release branch (and vice versa)

Done Criteria

• Code has been reviewed by other team members
• Analysis of existing tests (Unit and Integration)
• Unit Tests created for new code or existing Unit Tests updated
• Integration Test Suite updated (includes deletion of existing unnecessary Integration Test and/or creation of new ones if required)
• Enduser Documentation updated (if applicable)
• Internal technical Documentation created/updated (if applicable)
• Successful demonstration in Review

[frewilhelm]

As a note for discussions:

PR about CLI => Run with PR CLI + Main Controller
PR about Controller => Run with PR Controller + Main CLI
Main Push => Run with CLI + Controller Main
RC about CLI => Run with RC CLI + Release branch state
RC about Controller => Run with RC Controller + Release branch state

[Skarlso]

I have a couple options that I looked at in other repos, flux, crossplane, kubernetes, argo, cert-manager, and a few others.

• if we merge things together that would be a massive change and we would lose the ability do proper backports, release patches for individual components, or have different cadences for all our components ( meaning a bump in controller would bump the cli even if there are no changes in the cli ) ( cert-manager does this btw )
• we could leave things separate but go the route of providing a BOM / wrapper that is an umbrella that says, ok version 2.0 contains cli version 2.0.3 and controller version 2.3.4 or something and the conformance test runs against THIS COMBINATION only
• we could go with what we have today, but fix the actual problem with the ref and the main thing

[Skarlso]

HMMMMMMM.

Alllsoooo... here is a wild idea... We could drop conformance from the release completely. Main is already gated by it. In theory, nothing should be merged ever if conformance fails. 🤔

[Skarlso]

The more I look at it, the more I think we are double testing the same thing. Of course what we would lose is testing on backports. But I would take that risk. 🤔 Would reduce complexity, runtime and get rid of this pesky problem.

[Skarlso]

Okay, here is the decision:

We will have one release trigger with one release version in the end that will contain the individual components. We will drop the cli/* tag at that point, because it's useless, however, we are going to keep the kubernetes/controller/* tag, because that's genuinely being used by the folks who import our stuff now. ( we have two teams doing that already ).

We will still have individual flows, but a single trigger and a tag will make sure that the conformance test is running smoothly for all parties involved. This also means, that yes, we are going to release these modules together including patch versions. We have one release. One tag. That's it.

I'm going to try to have small prs. I think I can pull it off, because technically, I'll just make some reusable workflows, but let's see if I can even do some deduplication in the process.

[Skarlso]

As I'm doing this I noticed something else. Now, that I'm bringing the flows together, there is an optimization to be made if I pull the e2e test out and put it in the new pipeline I'm building. It will be able to run in parallel with the conformance test which saved considerable runtime. But I will do that as a follow-up because I don't want to clutter the review further.