[multicast,instance,test-flake] reorder instance_stop and send stop before tearing down multicast member state#10402
Conversation
…efore tearing down multicast member state Handles (closes) #9711 (was fixed downstream in a PR'ed branch).
|
@jgallagher |
This addresses a review comment on the synchronous `multicast_group_members_detach_by_instance` call introduced by the prior reorder, as a transient DB failure would 500 the caller for a stop that already succeeded at the sled, while short-circuiting past the reconciler activation. The state would be self-healing via the reconciler, but the visible failure for a successful op would be a regression. Instead, we move the detach into the `instance_update` saga's `siu_commit_instance_updates` action, gated on `update.deprovision.is_some()`, where the saga signals an instance has reached no-active-VMM state. That saga already orchestrates terminal-VMM cleanup, so the detach fits naturally there. As a side effect, this also covers guest-initiated shutdown and sled-agent-reported failure paths that the `instance_stop` callsite hadn't covered. And, we still have the reconciler to check through things. This change also includes a reconciler nudge in `instance_stop` in the case where the saga does not fire if there were no terminal VMM transition to drive it in the first place (instead of waiting for the full reconciler pass on next tick).
|
This included a merge of main over the top of the fixes. |
jgallagher
left a comment
jgallagher
left a comment
There was a problem hiding this comment.
I don't do much with instance sagas or multicast, but the changes LGTM and definitely look more correct than the original version. Happy to approve based on your testing, but I won't be offended if you want to wait for an approval from someone else with more context.
| .await | ||
| { | ||
| info!(log, | ||
| "instance update: failed to detach multicast members on deprovision, next reconciler pass will retry"; |
There was a problem hiding this comment.
Trivial nit with my apologies - mind wrapping this to 80 columns? rustfmt doesn't try to split long strings, or do much with slog macros :(
There was a problem hiding this comment.
oh for sure (this is not a standard across the org repos, but I usually come back to these, so good catch).
2 participants
Handles (closes) #9711 (was fixed downstream in a PR'ed branch).