Kube-adaptor is not syncing resources when a secret is missing

[fgiorgetti]

Describe the bug
If you have a resource that depends on an externally provided tls credential (generateTlsCredentials: false), like a RouterAccess, and the respective secret is not available, kube-adaptor is not proceeding with the syncronization logic for the other entities, like bridges and links.

How To Reproduce

1. Create a RouterAccess resource with .spec.generateTlsCredentials: false.
2. Create any other resource like, a listener: skupper listener create backend 8080.
3. List the active tcpListeners, using: kubectl exec deploy/skupper-router -- skmanage query --type tcpListener

You will see an empty list of tcpListeners as they have been defined to the ConfigMap, but not through the Management API.

Expected behavior
If a resource, like RouterAccess, Listener, Connector refers to a TLS Credential that does not exist, they should simply be skipped from the router configuration at all (including the configmap itself), so that the configuration of other entities are not impacted.

In case the router pod restarts, the config-init container will prevent it from starting, with the following error:

Waiting for Secrets to be created for SslProfiles

Environment details

• Skupper 2.1.3

[c-kruse]

@AryanP123 I think the direction you're taking in the PR is really close to the direction I think is the right way forward. Example #2107 which calls out not configuring the routeraccess AND updating RouterAccess status with a reason: pretty much what I see you up to with the Link resource.

I think the direction @fgiorgetti had in mind was a more targeted fix for how kube-adapter behaves when it's put in this situation (that it ideally would not be put in.)

[fgiorgetti]

@c-kruse @AryanP123 we need to prevent the controller from writing entities into the router configmaps if the respective secrets are not yet available.

We should not prevent the controller from reconciling the CRs, as for example, the hostnames or IPs of a RouterAccess must be resolved, before the server certificate can be generated.

But the resulting listener, from a RouterAccess in READY state, should not be persisted into the router configmaps, as it could leave the config-init container hanging for an eventual certificate that might never show up.

One comment @AryanP123 is that your PR is currently getting secrets from the API (through: tlsCredentialSecretPresent) when validating Links, Bindings and RouterAccesses. Maybe better to use cached secrets from ProfilesWatcher or something exclusive.

[c-kruse]

+1 @fgiorgetti

I've got to get reacquainted with the site controller logic here, I recall that some of the patterns we use really work against us here. Once upon a time I took a stab at cleaning this up and I remember it turning into a pretty large overhaul. Maybe we should do some sort of design step before digging in - I think it could be larger than just tlsCredentials Secrets: ex the proxy profile Secret.

IIRC I was looking at a more general facility for configuration dependencies (and a path to re-queue related resources when those dependencies are realized, deleted or maybe changed), and a standard template for reconciling site resources, something like: compare desired and actual states, remediate any pre-configuration deltas, check configuration dependencies, apply router configuration, set as ready/configured.

[fgiorgetti]

@c-kruse I agree, we could use this as an opportunity to re-design this whole configuration mechanism to include dependency checks before applying configuration, not limited just to the ssl profiles.
With the upcoming changes related to multi-van we will start to manage new router entities that are not being managed today, like the autoLinks, which also depends on an existing listener or connector and eventually on the network.networkId.

Even a basicy dependency check done at qdr.RouterConfig.WriteToConfigMap could be helpful to prevent leaving the router config in a bad state.