Skip to content

Commit 1212634

Browse files
lmicciniclaude
andcommitted
Immutable secrets and consumer finalizers for TransportURL credential rotation
Implement safe credential rotation for TransportURL by creating immutable secrets with content-hashed names, adding per-consumer finalizers to coordinate lifecycle between TransportURL and RabbitMQUser CRs, and gating old user release on both consumer finalizer removal and NodeSet secret hash synchronization. Key changes: - Create immutable transport secrets (rabbitmq-transport-url-{name}-{hash}) during rotation to prevent content mutation by consumers - Add per-consumer finalizers (turl.openstack.org/t-{name}) on shared RabbitMQUser and RabbitMQVhost CRs to track active consumers - Add transport secret consumer finalizer protocol for consuming operators to signal rollout completion - Unified release path: wait for consumer finalizer removal, then check NodeSet secret hash sync — if hashes are in sync the secret is not tracked by the dataplane and the old user is released immediately; if out of sync, wait for full NodeSet deployment to complete - Prevent SecretName flip-flop by comparing content hashes before deciding whether to create a new immutable secret - Auto-delete orphaned RabbitMQUser CRs when all consumers release their finalizers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 5a4764b commit 1212634

8 files changed

Lines changed: 597 additions & 241 deletions

File tree

apis/bases/rabbitmq.openstack.org_transporturls.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,12 @@ spec:
132132
finalizer removal was deferred during a credential rotation, pending
133133
deployment verification. Only set while a rotation is in progress.
134134
type: string
135+
previousSecretName:
136+
description: |-
137+
PreviousSecretName - name of the previous TransportURL secret retained
138+
during credential rotation, pending consumer finalization.
139+
Cleared when the old secret is cleaned up after all consumers release it.
140+
type: string
135141
queueType:
136142
description: QueueType - the queue type from the associated RabbitMq
137143
instance
@@ -153,6 +159,12 @@ spec:
153159
rabbitmqVhost:
154160
description: RabbitmqVhost - the actual vhost name used
155161
type: string
162+
secretHash:
163+
description: |-
164+
SecretHash - hash of the current TransportURL secret content.
165+
Consuming services compare this with the hash they computed to
166+
confirm credential rotation is complete.
167+
type: string
156168
secretName:
157169
description: SecretName - name of the secret containing the rabbitmq
158170
transport URL

apis/rabbitmq/v1beta1/conditions.go

Lines changed: 9 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -62,28 +62,17 @@ const (
6262
// completion, so the user controller can safely auto-delete the CR on sight.
6363
RabbitMQUserOrphanedLabel = "rabbitmq.openstack.org/orphaned"
6464

65-
// EDPMServiceAnnotation overrides the automatic owner-based EDPM detection
66-
// for a TransportURL. When set to "true", the controller always uses the
67-
// two-phase NodeSet sync check. When "false", it releases the old user
68-
// immediately. When unset, the controller infers EDPM status from the
69-
// ownerReference Kind (Nova and NeutronAPI are EDPM; everything else is not).
70-
EDPMServiceAnnotation = "rabbitmq.openstack.org/edpm-service"
65+
// TransportSecretProtectionFinalizer prevents a transport URL secret from
66+
// being deleted while it is referenced by a TransportURL (current or previous).
67+
// Added atomically at secret creation; removed during cleanup.
68+
TransportSecretProtectionFinalizer = "openstack.org/transport-secret-protection"
69+
70+
// TransportSecretConsumerSuffix is the suffix appended to the operator name
71+
// to form the consumer finalizer on transport URL secrets.
72+
// Example: "openstack.org/keystone-transport-consumer"
73+
TransportSecretConsumerSuffix = "-transport-consumer"
7174
)
7275

73-
// edpmOwnerKinds lists the owner CR Kinds whose TransportURLs serve
74-
// EDPM-deployed agents and therefore require NodeSet hash-sync gating
75-
// during credential rotation.
76-
var edpmOwnerKinds = map[string]bool{
77-
"Nova": true,
78-
"NeutronAPI": true,
79-
}
80-
81-
// IsEDPMOwnerKind reports whether the given owner Kind serves
82-
// EDPM-deployed agents that require NodeSet hash-sync gating.
83-
func IsEDPMOwnerKind(kind string) bool {
84-
return edpmOwnerKinds[kind]
85-
}
86-
8776
// TransportURLFinalizerFor returns the per-consumer finalizer for a TransportURL.
8877
// If the name fits within Kubernetes' 63-char name segment limit, it is used directly
8978
// (preserving human readability and reverse mapping). For longer names, the suffix
Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
/*
2+
Licensed under the Apache License, Version 2.0 (the "License");
3+
you may not use this file except in compliance with the License.
4+
You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software
9+
distributed under the License is distributed on an "AS IS" BASIS,
10+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
See the License for the specific language governing permissions and
12+
limitations under the License.
13+
*/
14+
15+
package v1beta1
16+
17+
import (
18+
"context"
19+
"fmt"
20+
"strings"
21+
22+
"github.com/openstack-k8s-operators/lib-common/modules/common/helper"
23+
"github.com/openstack-k8s-operators/lib-common/modules/common/object"
24+
corev1 "k8s.io/api/core/v1"
25+
k8s_errors "k8s.io/apimachinery/pkg/api/errors"
26+
"k8s.io/apimachinery/pkg/types"
27+
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
28+
)
29+
30+
// ManageTransportSecretFinalizer ensures consumerFinalizer is present on the
31+
// transport secret identified by secretName. It never removes the finalizer
32+
// from a previous secret — that is the consumer's responsibility after it has
33+
// confirmed its deployment is running with the new credentials (typically via
34+
// RemoveTransportSecretConsumerFinalizer). This ensures the TransportURL
35+
// controller waits for all consumers before releasing the old RabbitMQ user.
36+
func ManageTransportSecretFinalizer(
37+
ctx context.Context,
38+
h *helper.Helper,
39+
namespace string,
40+
secretName string,
41+
consumerFinalizer string,
42+
) error {
43+
if secretName == "" {
44+
return nil
45+
}
46+
47+
secret := &corev1.Secret{}
48+
key := types.NamespacedName{Name: secretName, Namespace: namespace}
49+
if err := h.GetClient().Get(ctx, key, secret); err != nil {
50+
return fmt.Errorf("failed to get transport secret %s: %w", secretName, err)
51+
}
52+
53+
return object.AddConsumerFinalizer(ctx, h, secret, consumerFinalizer)
54+
}
55+
56+
// RemoveTransportSecretConsumerFinalizer removes consumerFinalizer from the
57+
// transport secret identified by secretName. It is a no-op when secretName
58+
// is empty or the secret no longer exists.
59+
func RemoveTransportSecretConsumerFinalizer(
60+
ctx context.Context,
61+
h *helper.Helper,
62+
namespace string,
63+
secretName string,
64+
consumerFinalizer string,
65+
) error {
66+
if secretName == "" {
67+
return nil
68+
}
69+
70+
secret := &corev1.Secret{}
71+
key := types.NamespacedName{Name: secretName, Namespace: namespace}
72+
if err := h.GetClient().Get(ctx, key, secret); err != nil {
73+
if k8s_errors.IsNotFound(err) {
74+
return nil
75+
}
76+
return err
77+
}
78+
return object.RemoveConsumerFinalizer(ctx, h, secret, consumerFinalizer)
79+
}
80+
81+
// HasTransportConsumerFinalizer returns true if the secret has any finalizer
82+
// matching the transport consumer pattern (openstack.org/*-transport-consumer).
83+
func HasTransportConsumerFinalizer(secret *corev1.Secret) bool {
84+
for _, f := range secret.Finalizers {
85+
if strings.HasSuffix(f, TransportSecretConsumerSuffix) &&
86+
strings.HasPrefix(f, "openstack.org/") {
87+
return true
88+
}
89+
}
90+
return false
91+
}
92+
93+
// HasSpecificTransportConsumerFinalizer returns true if the secret has the
94+
// given consumer finalizer.
95+
func HasSpecificTransportConsumerFinalizer(secret *corev1.Secret, consumerFinalizer string) bool {
96+
return controllerutil.ContainsFinalizer(secret, consumerFinalizer)
97+
}

apis/rabbitmq/v1beta1/transporturl_types.go

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,16 @@ type TransportURLStatus struct {
6666
// Empty if using default cluster admin credentials (no dedicated RabbitMQUser CR)
6767
RabbitmqUserRef string `json:"rabbitmqUserRef,omitempty"`
6868

69+
// SecretHash - hash of the current TransportURL secret content.
70+
// Consuming services compare this with the hash they computed to
71+
// confirm credential rotation is complete.
72+
SecretHash string `json:"secretHash,omitempty"`
73+
74+
// PreviousSecretName - name of the previous TransportURL secret retained
75+
// during credential rotation, pending consumer finalization.
76+
// Cleared when the old secret is cleaned up after all consumers release it.
77+
PreviousSecretName string `json:"previousSecretName,omitempty"`
78+
6979
// PreviousRabbitmqUserRef - the name of a previous RabbitMQUser CR whose
7080
// finalizer removal was deferred during a credential rotation, pending
7181
// deployment verification. Only set while a rotation is in progress.

config/crd/bases/rabbitmq.openstack.org_transporturls.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,12 @@ spec:
132132
finalizer removal was deferred during a credential rotation, pending
133133
deployment verification. Only set while a rotation is in progress.
134134
type: string
135+
previousSecretName:
136+
description: |-
137+
PreviousSecretName - name of the previous TransportURL secret retained
138+
during credential rotation, pending consumer finalization.
139+
Cleared when the old secret is cleaned up after all consumers release it.
140+
type: string
135141
queueType:
136142
description: QueueType - the queue type from the associated RabbitMq
137143
instance
@@ -153,6 +159,12 @@ spec:
153159
rabbitmqVhost:
154160
description: RabbitmqVhost - the actual vhost name used
155161
type: string
162+
secretHash:
163+
description: |-
164+
SecretHash - hash of the current TransportURL secret content.
165+
Consuming services compare this with the hash they computed to
166+
confirm credential rotation is complete.
167+
type: string
156168
secretName:
157169
description: SecretName - name of the secret containing the rabbitmq
158170
transport URL

config/rbac/role.yaml

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -73,29 +73,6 @@ rules:
7373
- patch
7474
- update
7575
- watch
76-
- apiGroups:
77-
- barbican.openstack.org
78-
- cinder.openstack.org
79-
- designate.openstack.org
80-
- glance.openstack.org
81-
- heat.openstack.org
82-
- horizon.openstack.org
83-
- ironic.openstack.org
84-
- keystone.openstack.org
85-
- manila.openstack.org
86-
- neutron.openstack.org
87-
- nova.openstack.org
88-
- octavia.openstack.org
89-
- ovn.openstack.org
90-
- placement.openstack.org
91-
- swift.openstack.org
92-
- telemetry.openstack.org
93-
- watcher.openstack.org
94-
resources:
95-
- '*'
96-
verbs:
97-
- get
98-
- list
9976
- apiGroups:
10077
- config.openshift.io
10178
resources:

0 commit comments

Comments
 (0)