Skip to content

Commit 71bf4d3

Browse files
kvapsclaude
andcommitted
fix(satellite): wipe stale .res files in state-dir on startup
drbdadm chokes on the WHOLE /etc/drbd.d/ if any single .res file in it has a parse error — even when the new RD's render is clean. Stand ran into this after a controller restart: stale .res files from a prior csi-sanity run carried `AutoEvictAllowEviction` (a Node prop that shouldn't be in DRBD options{}), and every fresh smoke RD's drbdadm primary failed with a parse error from the unrelated stale file. Wipe *.res in --state-dir on satellite startup; the controller re-Applies every Resource CRD on this node within seconds of Hello, so the contents are reproducible. global_common.conf is left intact. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
1 parent 5de5f6a commit 71bf4d3

1 file changed

Lines changed: 55 additions & 0 deletions

File tree

cmd/satellite/main.go

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ import (
2828
"log/slog"
2929
"os"
3030
"os/signal"
31+
"path/filepath"
32+
"strings"
3133
"syscall"
3234
"time"
3335

@@ -136,6 +138,17 @@ func run() int {
136138
storage.RealExec{})
137139
}
138140

141+
// Wipe stale .res files from previous incarnations of this
142+
// satellite process. Each pod restart should hand drbdadm a
143+
// clean slate — the controller will re-Apply every Resource
144+
// CRD on this node within seconds, so we don't lose state, just
145+
// the cruft from prior runs (csi-sanity leftovers, RDs deleted
146+
// while the satellite was down, malformed .res from earlier
147+
// release versions, etc). Without this drbdadm fails on parse
148+
// errors from any one stale file even when the new RD's render
149+
// is clean.
150+
cleanStateDir(stateDir, logger)
151+
139152
agent := satellite.NewAgent(satellite.Config{
140153
NodeName: nodeName,
141154
ControllerAddr: controllerAddr,
@@ -168,3 +181,45 @@ func run() int {
168181

169182
return 0
170183
}
184+
185+
// cleanStateDir wipes every *.res file in dir on satellite startup.
186+
// The controller re-Applies every Resource CRD on this node shortly
187+
// after Hello, so the contents are reproducible — we don't persist
188+
// satellite-side state across restarts. Best-effort: log and continue
189+
// on errors so a single missing dir doesn't stall the whole startup.
190+
func cleanStateDir(dir string, logger *slog.Logger) {
191+
entries, err := os.ReadDir(dir)
192+
if err != nil {
193+
// Missing dir is fine — the satellite's first Apply will
194+
// create it on demand.
195+
return
196+
}
197+
198+
removed := 0
199+
200+
for _, entry := range entries {
201+
if entry.IsDir() {
202+
continue
203+
}
204+
205+
name := entry.Name()
206+
if !strings.HasSuffix(name, ".res") {
207+
// Leave global_common.conf and any operator-supplied
208+
// non-rendered files alone.
209+
continue
210+
}
211+
212+
path := filepath.Join(dir, name)
213+
if err := os.Remove(path); err != nil {
214+
logger.Warn("clean state-dir entry", "path", path, "err", err)
215+
216+
continue
217+
}
218+
219+
removed++
220+
}
221+
222+
if removed > 0 {
223+
logger.Info("wiped stale .res files on startup", "dir", dir, "removed", removed)
224+
}
225+
}

0 commit comments

Comments
 (0)