[mcast] audit-pass + reconciler updates + opte hardening + config tuning

Final, pre-review pass on this work. It stacks atop #10070 and inherits
the multicast-to-physical (M2P) underlay forwarding and VMM-keyed instance
subscription endpoints.

This also builds on and integrates #10381.

Above these foundations, this work includes the final pass on mgd-ddmd
integration:

* Reconciler correctness:
  * `set_mcast_m2p` rolls back the xde M2P entry on per-NIC join
    failure, so the reconciler converges on a retry instead of
    leaving stale state pointing at the wrong underlay address.
  * `propolis_id` is threaded end-to-end through the sled-agent
    multicast endpoints to deal with live migration ambiguity.
  * MRIB advertisement is gated on a flag rather than running unconditionally
    after the DPD match arm, so that a DPD failure no longer leaves
    a route advertised via DDM with no programmed forwarding state.

* OPTE hardening (illumos-utils):
  * M2P entries upserted into a `BTreeMap<IpAddr, MulticastUnderlay>`
    rather than a Vec on the non-illumos mock, eliminating duplicate-key corner
    cases the production map already avoided.
  * `MulticastFilterMap` encapsulates the per-NIC filter socket and
    refcount state previously open-coded inside `PortManagerInner`,
    concentrating the "join socket per underlay group per NIC"
    invariant into one singular type.
  * underlay_nics typed as &[AddrObject] rather than &[String].
  * Per-NIC IPV6_JOIN_GROUP calls converted from libc::setsockopt to
    nix::sys::socket::setsockopt for the typed bind.

* Sled-agent (real and sim):
  * Sim v7 multicast endpoints fall through to the trait defaults
    instead of overriding with just `unimplemented!()`, matching how
    other versioned endpoints behave in the sim.
  * Sim VMM existence check on join/leave restored.

* Configuration:
  * `MulticastGroupReconcilerConfig` gains a group_concurrency_limit
    and member_concurrency_limit bounding the per-pass fan-out of the RPW's
    buffer_unordered streams.

* Test infra:
  * `populate_ddm_peers` no longer caches the peer map. The previous
    cache was keyed by sled-id set, but the synthesized port names
    embedded each sled's `sp_slot` from inventory, so cache reuse
    within the same sled set could produce stale port mappings.

* Documentation cleanup across the RPW, sled-agent multicast paths, and the
  new(er) sled-agent types module.

Author: zeeshanlakhani

clients/ddm-admin-client/src/lib.rs

@@ -2,8 +2,6 @@
 // License, v. 2.0. If a copy of the MPL was not distributed with this
 // file, You can obtain one at https://mozilla.org/MPL/2.0/.
 
-// Copyright 2026 Oxide Computer Company
-
 #![allow(clippy::redundant_closure_call)]
 #![allow(clippy::needless_lifetimes)]
 #![allow(clippy::match_single_binding)]

illumos-utils/src/opte/illumos.rs

@@ -76,6 +76,12 @@ pub enum Error {
         "address {0} is not within the underlay multicast subnet (ff04::/16)"
     )]
     InvalidMcastUnderlay(Ipv6Addr),
+
+    #[error(
+        "failed to install NIC multicast MAC filter for underlay {0}, \
+         caller should retry"
+    )]
+    UnderlayMcastJoinFailed(Ipv6Addr),
 }
 
 /// Delete all xde devices on the system.

illumos-utils/src/opte/mod.rs

@@ -45,7 +45,7 @@ use std::net::Ipv6Addr;
 // `omicron_common::api::external::Vni::DEFAULT_MULTICAST_VNI` live in sibling
 // crates that cannot reference each other's constant. They must stay
 // numerically equal: the MRIB, M2P mappings, and OPTE all route on this
-// value, so any divergence would black-hole multicast traffic.
+// value, so any divergence would silently drop multicast traffic.
 const _: () = assert!(
     oxide_vpc::api::DEFAULT_MULTICAST_VNI
         == omicron_common::api::external::Vni::DEFAULT_MULTICAST_VNI.as_u32(),

illumos-utils/src/opte/non_illumos.rs

@@ -4,7 +4,7 @@
 
 //! Mock / dummy versions of the OPTE module, for non-illumos platforms.
 //!
-//! Most methods are either `unimplemented!()` or silent no-ops.
+//! Most methods are either `unimplemented!()` or silent noops.
 //! Multicast subscribe/unsubscribe is an exception, as it maintains real
 //! in-memory state because port manager tests assert on subscription contents.
 
@@ -37,6 +37,7 @@ use oxide_vpc::api::SourceFilter;
 use oxide_vpc::api::VpcCfg;
 use sled_agent_types::inventory::NetworkInterfaceKind;
 use slog::Logger;
+use std::collections::BTreeMap;
 use std::collections::HashMap;
 use std::collections::hash_map::Entry;
 use std::net::IpAddr;
@@ -94,6 +95,12 @@ pub enum Error {
         "address {0} is not within the underlay multicast subnet (ff04::/16)"
     )]
     InvalidMcastUnderlay(std::net::Ipv6Addr),
+
+    #[error(
+        "failed to install NIC multicast MAC filter for underlay {0}, \
+         caller should retry"
+    )]
+    UnderlayMcastJoinFailed(std::net::Ipv6Addr),
 }
 
 pub fn initialize_xde_driver(
@@ -198,11 +205,12 @@ pub(crate) struct PortData {
 pub(crate) struct State {
     pub ports: HashMap<String, PortData>,
     pub underlay_initialized: bool,
-    /// Multicast-to-physical mappings, keyed on (group, underlay).
+    /// Multicast-to-physical mappings, keyed by group.
     ///
     /// Persisted across [`Handle`] lifetimes to simulate xde kernel state
-    /// surviving sled-agent restarts.
-    pub m2p: Vec<(oxide_vpc::api::IpAddr, MulticastUnderlay)>,
+    /// surviving sled-agent restarts. Mirrors the upsert-by-group
+    /// semantics of xde's `Mcast2Phys`.
+    pub m2p: BTreeMap<oxide_vpc::api::IpAddr, MulticastUnderlay>,
 }
 
 const NO_RESPONSE: NoResp = NoResp { unused: 99 };
@@ -213,7 +221,7 @@ fn opte_state() -> &'static Mutex<State> {
         Mutex::new(State {
             ports: HashMap::new(),
             underlay_initialized: false,
-            m2p: Vec::new(),
+            m2p: BTreeMap::new(),
         })
     })
 }
@@ -391,9 +399,8 @@ impl Handle {
     /// Set a multicast-to-physical mapping.
     pub fn set_m2p(&self, req: &SetMcast2PhysReq) -> Result<NoResp, OpteError> {
         let mut state = opte_state().lock().unwrap();
-        // Deduplicate by replacing existing entry for the same group.
-        state.m2p.retain(|(g, _)| *g != req.group);
-        state.m2p.push((req.group, req.underlay));
+        // Upsert: replaces any existing entry for the same group.
+        state.m2p.insert(req.group, req.underlay);
         Ok(NO_RESPONSE)
     }
 
@@ -403,7 +410,9 @@ impl Handle {
         req: &ClearMcast2PhysReq,
     ) -> Result<NoResp, OpteError> {
         let mut state = opte_state().lock().unwrap();
-        state.m2p.retain(|(g, u)| !(*g == req.group && *u == req.underlay));
+        if state.m2p.get(&req.group) == Some(&req.underlay) {
+            state.m2p.remove(&req.group);
+        }
         Ok(NO_RESPONSE)
     }