Skip to content

Commit f041f5d

Browse files
[rsc-controller] Add ReplicatedVolumeOperation API and rolling operations support
Introduce ReplicatedVolumeOperation (RVO) API for managing rolling configuration updates and eligible nodes conflict resolution. API changes: - Add ReplicatedVolumeOperation type with Spec (Type, ReplicatedVolumeName, ResolveEligibleNodesConflict) and Status (Conditions) - Add RollingOperationsCursor and SortSalt to ReplicatedStorageClassStatus - Add LastOperation field to ReplicatedVolumeStatus for UI observability - Add RSCRVOLockFinalizer and OperationCursorLabelKey constants RSC controller changes: - Add RVO watch with predicates (Completed condition changes) - Add RVO indexes (by RSC owner ref, by RV owner ref) - Implement selectTargetRollingOperations with filter-before-compute optimization (reduces hash computations from O(N) to O(C)) - Implement round-robin cursor-based volume selection with budget limits - Add removeRVOLockFinalizers for cleanup when cursor advances - Add defensive check for empty cursor label (migration safety) Performance: - Add go-performance-patterns.mdc rule with Schwartzian transform and filter-before-compute patterns Tests: - Add 36 new unit tests for rolling operations logic Signed-off-by: Ivan Ogurchenok <ivan.ogurchenok@flant.com>
1 parent 36f49a0 commit f041f5d

20 files changed

Lines changed: 1932 additions & 70 deletions
Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
---
2+
description: Go performance patterns for sorting and iteration. Apply when reviewing code with sort.Slice over collections that may have 100+ items with non-trivial comparators, or when expensive operations are applied to collections before filtering.
3+
alwaysApply: false
4+
---
5+
6+
See `rfc-like-mdc.mdc` for normative keywords (BCP 14 / RFC 2119 / RFC 8174) and general .mdc writing conventions.
7+
8+
# Go performance patterns
9+
10+
## Schwartzian transform for sorting (SHOULD)
11+
12+
When sorting slices where:
13+
- the comparator calls **expensive functions** (hash computations, string formatting, allocations), AND
14+
- the slice may contain **100+ items** in production scenarios,
15+
16+
you SHOULD use the **Schwartzian transform** (decorate-sort-undecorate) pattern:
17+
18+
1. **Decorate**: Precompute expensive values into a temporary struct slice — O(N)
19+
2. **Sort**: Sort by precomputed values (cheap comparison) — O(N log N) comparisons
20+
3. **Undecorate**: Extract results — O(N)
21+
22+
This reduces expensive computations from O(N log N) to O(N).
23+
24+
### Why it matters
25+
26+
| Items | Comparisons (sort) | Hash calls (naive) | Hash calls (optimized) |
27+
|-------|-------------------|--------------------|------------------------|
28+
| 100 | ~665 | ~1330 | 100 |
29+
| 500 | ~4483 | ~8966 | 500 |
30+
| 1000 | ~9966 | ~19932 | 1000 |
31+
32+
### Example (GOOD)
33+
34+
```go
35+
// sortItemsByExpensiveKey sorts items using Schwartzian transform.
36+
// Precomputes keys to avoid O(N log N) expensive calls.
37+
func sortItemsByExpensiveKey(items []Item) []Item {
38+
// Decorate: precompute keys once.
39+
type decorated struct {
40+
item *Item
41+
key string
42+
}
43+
tmp := make([]decorated, len(items))
44+
for i := range items {
45+
tmp[i] = decorated{
46+
item: &items[i],
47+
key: expensiveKeyComputation(items[i]),
48+
}
49+
}
50+
51+
// Sort by precomputed key.
52+
sort.SliceStable(tmp, func(i, j int) bool {
53+
return tmp[i].key < tmp[j].key
54+
})
55+
56+
// Undecorate: extract sorted items.
57+
result := make([]Item, len(tmp))
58+
for i, d := range tmp {
59+
result[i] = *d.item
60+
}
61+
return result
62+
}
63+
```
64+
65+
### Example (BAD)
66+
67+
```go
68+
// BAD: expensiveKeyComputation called O(N log N) times
69+
sort.SliceStable(items, func(i, j int) bool {
70+
return expensiveKeyComputation(items[i]) < expensiveKeyComputation(items[j])
71+
})
72+
```
73+
74+
### When reviewing code
75+
76+
When you see `sort.Slice` / `sort.SliceStable` with a non-trivial comparator:
77+
78+
1. ASK about expected collection size in production
79+
2. ASK about comparator cost (allocations, hashing, formatting)
80+
3. SUGGEST Schwartzian transform if:
81+
- Collection may have 100+ items, AND
82+
- Comparator is more expensive than simple field access
83+
84+
---
85+
86+
## Filter before compute (SHOULD)
87+
88+
When processing a collection with expensive per-element operations (hashing, serialization, complex computations), you SHOULD filter the collection **before** applying those operations.
89+
90+
### Rationale
91+
92+
If only M of N elements need processing, filtering first reduces expensive operations from O(N) to O(M).
93+
94+
### Example (BAD)
95+
96+
```go
97+
// BAD: computes expensive values for ALL N items, then filters
98+
sortedAll := sortByExpensiveHash(items) // N hash calls
99+
var candidates []Item
100+
for _, item := range sortedAll {
101+
if needsProcessing(item) { // cheap check
102+
candidates = append(candidates, item)
103+
}
104+
}
105+
// Only M candidates actually needed, but we computed N hashes
106+
```
107+
108+
### Example (GOOD)
109+
110+
```go
111+
// GOOD: filter first, then compute only for candidates
112+
var candidates []Item
113+
for i := range items {
114+
if needsProcessing(&items[i]) { // cheap check: O(N)
115+
candidates = append(candidates, items[i])
116+
}
117+
}
118+
// Now compute expensive values only for M candidates
119+
sortedCandidates := sortByExpensiveHash(candidates) // M hash calls
120+
```
121+
122+
### Combining with Schwartzian transform
123+
124+
When both patterns apply:
125+
126+
1. **Filter** to candidates — O(N) cheap checks
127+
2. **Decorate** candidates with expensive keys — O(M) expensive calls
128+
3. **Sort** by precomputed keys — O(M log M) cheap comparisons
129+
4. **Undecorate** (if needed) — O(M)
130+
131+
This reduces expensive computations from O(N + N log N) to O(M).
132+
133+
### When reviewing code
134+
135+
When you see expensive operations applied to a collection before filtering:
136+
137+
1. ASK: "Can we filter first to reduce the set?"
138+
2. CHECK: Is the filter condition cheap (simple field checks, map lookups)?
139+
3. SUGGEST filter-before-compute if:
140+
- Only a subset M of N items need the expensive operation, AND
141+
- The filter condition is significantly cheaper than the expensive operation

api/v1alpha1/finalizers.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,7 @@ const AgentFinalizer = "sds-replicated-volume.deckhouse.io/agent"
2121
const ControllerFinalizer = "sds-replicated-volume.deckhouse.io/controller"
2222

2323
const RSCControllerFinalizer = "sds-replicated-volume.deckhouse.io/rsc-controller"
24+
25+
// RSCRVOLockFinalizer is set by the RSC controller on RVO objects to prevent
26+
// garbage collection until the rolling update cursor has moved forward.
27+
const RSCRVOLockFinalizer = "sds-replicated-volume.deckhouse.io/rsc-rvo-lock"

api/v1alpha1/labels.go

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,4 +34,8 @@ const (
3434

3535
// AgentNodeLabelKey is the label key for selecting nodes where the agent should run.
3636
AgentNodeLabelKey = "storage.deckhouse.io/sds-replicated-volume-node"
37+
38+
// OperationCursorLabelKey is the label key for storing the cursor value on RVO objects.
39+
// Used to reliably identify which cursor the operation belongs to without parsing the name.
40+
OperationCursorLabelKey = labelPrefix + "operation-cursor"
3741
)

api/v1alpha1/register.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ func addKnownTypes(scheme *runtime.Scheme) error {
5252
&ReplicatedVolumeAttachmentList{},
5353
&ReplicatedVolumeReplica{},
5454
&ReplicatedVolumeReplicaList{},
55+
&ReplicatedVolumeOperation{},
56+
&ReplicatedVolumeOperationList{},
5557
&DRBDResource{},
5658
&DRBDResourceList{},
5759
&DRBDResourceOperation{},

api/v1alpha1/rsc_types.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -332,6 +332,15 @@ type ReplicatedStorageClassStatus struct {
332332
// Volumes provides aggregated volume statistics.
333333
// Always present (may have total=0).
334334
Volumes ReplicatedStorageClassVolumesSummary `json:"volumes"`
335+
336+
// SortSalt is a random value used for deterministic volume sorting during rolling operations.
337+
// Generated when rolling operations start, cleared when all volumes are aligned.
338+
// +optional
339+
SortSalt string `json:"sortSalt,omitempty"`
340+
// RollingOperationsCursor tracks the last processed volume hash during rolling operations.
341+
// Used for round-robin processing of volumes (both config updates and conflict resolutions).
342+
// +optional
343+
RollingOperationsCursor string `json:"rollingOperationsCursor,omitempty"`
335344
}
336345

337346
// ReplicatedStorageClassEligibleNodesWorldState tracks external state that affects eligible nodes.

api/v1alpha1/rv_types.go

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,11 @@ type ReplicatedVolumeStatus struct {
136136
// EligibleNodesViolations lists replicas placed on non-eligible nodes.
137137
// +optional
138138
EligibleNodesViolations []ReplicatedVolumeEligibleNodesViolation `json:"eligibleNodesViolations,omitempty"`
139+
140+
// LastOperation tracks the most recent rolling operation for this volume.
141+
// Provides observability for kubectl and UI.
142+
// +optional
143+
LastOperation *ReplicatedVolumeLastOperation `json:"lastOperation,omitempty"`
139144
}
140145

141146
// DeviceMinor is a DRBD device minor number.
@@ -290,3 +295,23 @@ const (
290295
)
291296

292297
func (r ReplicatedVolumeEligibleNodesViolationReason) String() string { return string(r) }
298+
299+
// ReplicatedVolumeLastOperation tracks the most recent rolling operation for a volume.
300+
// Provides observability for kubectl get and UI.
301+
// +kubebuilder:object:generate=true
302+
type ReplicatedVolumeLastOperation struct {
303+
// Type is the operation type (UpdateConfiguration or ResolveEligibleNodesConflict).
304+
Type string `json:"type"`
305+
// Name is the ReplicatedVolumeOperation resource name.
306+
Name string `json:"name"`
307+
// CreatedAt is when the operation was created.
308+
CreatedAt metav1.Time `json:"createdAt"`
309+
// CompletedAt is when the operation completed (nil if still in progress).
310+
// +optional
311+
CompletedAt *metav1.Time `json:"completedAt,omitempty"`
312+
// Status is the operation status: Pending, InProgress, Succeeded, or Failed.
313+
Status string `json:"status"`
314+
// Message provides additional information about the operation status.
315+
// +optional
316+
Message string `json:"message,omitempty"`
317+
}

api/v1alpha1/rvo_conditions.go

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
/*
2+
Copyright 2026 Flant JSC
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha1
18+
19+
const (
20+
// ReplicatedVolumeOperationCondCompletedType indicates whether the operation has completed.
21+
// When status is True, the operation finished (check reason for success/failure).
22+
// When status is False, the operation is still in progress or pending.
23+
//
24+
// Reasons describe completion state.
25+
ReplicatedVolumeOperationCondCompletedType = "Completed"
26+
ReplicatedVolumeOperationCondCompletedReasonFailed = "Failed" // Operation failed.
27+
ReplicatedVolumeOperationCondCompletedReasonInProgress = "InProgress" // Operation is being processed.
28+
ReplicatedVolumeOperationCondCompletedReasonPending = "Pending" // Operation is waiting to be processed.
29+
ReplicatedVolumeOperationCondCompletedReasonSucceeded = "Succeeded" // Operation completed successfully.
30+
ReplicatedVolumeOperationCondCompletedReasonVolumeGone = "VolumeGone" // Target volume no longer exists.
31+
)

api/v1alpha1/rvo_types.go

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
/*
2+
Copyright 2026 Flant JSC
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha1
18+
19+
import (
20+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
21+
)
22+
23+
// ReplicatedVolumeOperation represents an operation to update volume configuration
24+
// or resolve eligible nodes conflicts for a ReplicatedVolume.
25+
//
26+
// +kubebuilder:object:generate=true
27+
// +kubebuilder:object:root=true
28+
// +kubebuilder:subresource:status
29+
// +kubebuilder:resource:scope=Cluster,shortName=rvo
30+
// +kubebuilder:metadata:labels=module=sds-replicated-volume
31+
// +kubebuilder:printcolumn:name="Volume",type=string,JSONPath=".spec.replicatedVolumeName"
32+
// +kubebuilder:printcolumn:name="Type",type=string,JSONPath=".spec.type"
33+
// +kubebuilder:printcolumn:name="Completed",type=string,JSONPath=".status.conditions[?(@.type=='Completed')].status"
34+
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=".metadata.creationTimestamp"
35+
type ReplicatedVolumeOperation struct {
36+
metav1.TypeMeta `json:",inline"`
37+
metav1.ObjectMeta `json:"metadata"`
38+
39+
Spec ReplicatedVolumeOperationSpec `json:"spec"`
40+
// +patchStrategy=merge
41+
Status ReplicatedVolumeOperationStatus `json:"status,omitempty" patchStrategy:"merge"`
42+
}
43+
44+
// +kubebuilder:object:generate=true
45+
// +kubebuilder:object:root=true
46+
// +kubebuilder:resource:scope=Cluster
47+
type ReplicatedVolumeOperationList struct {
48+
metav1.TypeMeta `json:",inline"`
49+
metav1.ListMeta `json:"metadata"`
50+
Items []ReplicatedVolumeOperation `json:"items"`
51+
}
52+
53+
// GetStatusConditions is an adapter method to satisfy objutilv1.StatusConditionObject.
54+
// It returns the root object's `.status.conditions`.
55+
func (o *ReplicatedVolumeOperation) GetStatusConditions() []metav1.Condition {
56+
return o.Status.Conditions
57+
}
58+
59+
// SetStatusConditions is an adapter method to satisfy objutilv1.StatusConditionObject.
60+
// It sets the root object's `.status.conditions`.
61+
func (o *ReplicatedVolumeOperation) SetStatusConditions(conditions []metav1.Condition) {
62+
o.Status.Conditions = conditions
63+
}
64+
65+
// ReplicatedVolumeOperationSpec defines the desired state of a ReplicatedVolumeOperation.
66+
// All fields are immutable after creation.
67+
//
68+
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="spec is immutable"
69+
// +kubebuilder:object:generate=true
70+
type ReplicatedVolumeOperationSpec struct {
71+
// ReplicatedVolumeName is the name of the ReplicatedVolume this operation targets.
72+
// +kubebuilder:validation:Required
73+
// +kubebuilder:validation:MinLength=1
74+
// +kubebuilder:validation:MaxLength=120
75+
ReplicatedVolumeName string `json:"replicatedVolumeName"`
76+
77+
// Type specifies the operation type.
78+
// +kubebuilder:validation:Required
79+
// +kubebuilder:validation:Enum=UpdateConfiguration;ResolveEligibleNodesConflict
80+
Type ReplicatedVolumeOperationType `json:"type"`
81+
82+
// ResolveEligibleNodesConflict indicates whether this operation should also resolve
83+
// eligible nodes conflicts when updating configuration. Only meaningful when Type is
84+
// UpdateConfiguration. Determined by EligibleNodesConflictResolutionStrategy.
85+
// +kubebuilder:default=false
86+
// +optional
87+
ResolveEligibleNodesConflict bool `json:"resolveEligibleNodesConflict,omitempty"`
88+
}
89+
90+
// ReplicatedVolumeOperationType enumerates possible operation types.
91+
type ReplicatedVolumeOperationType string
92+
93+
const (
94+
// ReplicatedVolumeOperationTypeUpdateConfiguration updates the volume configuration
95+
// to match the current storage class configuration.
96+
ReplicatedVolumeOperationTypeUpdateConfiguration ReplicatedVolumeOperationType = "UpdateConfiguration"
97+
// ReplicatedVolumeOperationTypeResolveEligibleNodesConflict moves replicas
98+
// from non-eligible nodes to eligible nodes.
99+
ReplicatedVolumeOperationTypeResolveEligibleNodesConflict ReplicatedVolumeOperationType = "ResolveEligibleNodesConflict"
100+
)
101+
102+
func (t ReplicatedVolumeOperationType) String() string { return string(t) }
103+
104+
// Short name suffixes for operation resource naming.
105+
// Format: <rv-name>-<suffix>-<cursor>
106+
const (
107+
// ReplicatedVolumeOperationNameSuffixUpdateConfig is the short suffix for UpdateConfiguration operations.
108+
ReplicatedVolumeOperationNameSuffixUpdateConfig = "updateConfig"
109+
// ReplicatedVolumeOperationNameSuffixResolveNodes is the short suffix for ResolveEligibleNodesConflict operations.
110+
ReplicatedVolumeOperationNameSuffixResolveNodes = "resolveNodes"
111+
)
112+
113+
// NameSuffix returns the short suffix for generating operation resource names.
114+
func (t ReplicatedVolumeOperationType) NameSuffix() string {
115+
switch t {
116+
case ReplicatedVolumeOperationTypeUpdateConfiguration:
117+
return ReplicatedVolumeOperationNameSuffixUpdateConfig
118+
case ReplicatedVolumeOperationTypeResolveEligibleNodesConflict:
119+
return ReplicatedVolumeOperationNameSuffixResolveNodes
120+
default:
121+
return string(t)
122+
}
123+
}
124+
125+
// ReplicatedVolumeOperationStatus represents the current state of a ReplicatedVolumeOperation.
126+
// +kubebuilder:object:generate=true
127+
type ReplicatedVolumeOperationStatus struct {
128+
// +patchMergeKey=type
129+
// +patchStrategy=merge
130+
// +listType=map
131+
// +listMapKey=type
132+
// +optional
133+
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
134+
}

0 commit comments

Comments
 (0)