Drop support for `otelcol.v1alpha1.opentelemetry.io`

[swiatekm]

We've released the v1beta1 version of the OpenTelemetryCollector CRD more than a year ago. It's time to deprecate and drop support for v1alpha1.

The plan is to use a feature gate to disable the conversion webhook, then remove it and the actual CRD from the manifest.

• vX Add feature gate, disabled by default, pin this issue.
• vX+1 Enable feature gate by default.
• vX+2 Set served: false on v1alpha1
• vX+N Set feature gate to Stable, can't be disabled anymore.
• vX+N+1 Drop feature gate, remove v1alpha1 from the CRD.

[pavolloffay]

I think we still have customers on v1alpha1. I am fine with removal, but I would prefer a longer time frame. Let's discuss it at the SIG meeting.

[swiatekm]

I think we still have customers on v1alpha1. I am fine with removal, but I would prefer a longer time frame. Let's discuss it at the SIG meeting.

Longer time frame before we flip the feature gate, or after we flip it but before we remove it?

[pavolloffay]

This is also a duplicate of #2727

The issue proposes to change the v1alpha1 to served: false. I am not sure what implications it has, but it's something we should explore.

[swiatekm]

This is also a duplicate of #2727

The issue proposes to change the v1alpha1 to served: false. I am not sure what implications it has, but it's something we should explore.

Reading the description in https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#overview

Removing an old version:

Ensure all clients are fully migrated to the new version. The kube-apiserver logs can be reviewed to help identify any clients that are still accessing via the old version.
Set served to false for the old version in the spec.versions list. If any clients are still unexpectedly using the old version they may begin reporting errors attempting to access the custom resource objects at the old version. If this occurs, switch back to using served:true on the old version, migrate the remaining clients to the new version and repeat this step.
Ensure the upgrade of existing objects to the new stored version step has been completed.
Verify that the storage is set to true for the new version in the spec.versions list in the CustomResourceDefinition.
Verify that the old version is no longer listed in the CustomResourceDefinition status.storedVersions.
Remove the old version from the CustomResourceDefinition spec.versions list.
Drop conversion support for the old version in conversion webhooks.

It sounds like this is basically making the API unusuable without actually removing the struct from the CRD. So it's something we should do as the feature gate moves to stable, before we completely remove the version from the CRD.

[ps-mir]

Setting served: false is fine. It will cause 404 on the v1alpha1 API, forcing users to migrate to the v1beta1 API, but they may skip the storage version migration.

When the CRD and API are eventually dropped, objects still stored as v1alpha1 would be silently lost.

One option would be to fail the deployment (for the dropping step) if v1alpha1 is still found in storedVersions, giving users a hard signal before data is affected.

K8s v1.35 has a storage version migration feature that could automate this, but it's currently beta and disabled by default:
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/storage-version-migration/

[swiatekm]

Making the operator exit with an error on startup if the CRD has v1alpha1 in status.storedVersion is a pretty good idea, we should incorporate that. It'll give users more heads-up on what they need to do to prepare their environment for the v1alpha1 removal.

served: false we should only do once the flag becomes stable, as it's a CRD attribute and much more involved to reverse vs just changing operator flags.

[ps-mir]

Both actions together disruptively cut off access to stored v1alpha1 versions without deleting them.

It would be preferable to start with the less complex CRD change. If the conversion webhook is disabled first, users also have path to switch back to the v1alpha1 api rather than doing the stored Version migration.

With served: false first, all users are uniformly pushed towards v1beta1 API, if not the complete stored version migration, which can be followed by disabling conversion webhook.

[swiatekm]

served: false is somewhat difficult to revert, since the user needs to patch the CRD. The whole point of the feature gate is to make it easy to re-enable support without needing to roll back the operator upgrade.

[ps-mir]

ok, that makes more sense, I was confused by the requirement

v0.154.0 Set feature gate to Stable, can't be disabled anymore.

I thought the feature gate couldn't change, and the CRD change will apply after this.

I was wondering how this applies to OLM. There, it seems every change is based on bundle version.

[swiatekm]

It can't change after it goes to Stable, but before that it can be changed. The idea is that after we enable it by default (set it to Beta), users will see the errors, turn off the feature gate, resolve the errors, turn it on, and be done with the migration.

[ps-mir]

If disabling the conversion webhook is sufficient for all use cases to nudge towards the v1beta1 stored version, then it should be fine.

As a proactive step, it would be good to fail the deployment (for the dropping step) if v1alpha1 is still found in storedVersions
before the feature gate is made stable.