Skip to content

Ubuntu marketplace images ship with NetworkManager installed (sets up dual-renderer conflict) #356

Description

@aioue

Summary

The Ubuntu marketplace appliances ship with network-manager installed and enabled, even though contextualization configures networking via netplan with the networkd renderer. On a freshly contextualized Ubuntu 26.04 VM, NetworkManager manages no interfaces and nothing depends on it. Having both NetworkManager and systemd-networkd active is a dual-renderer setup that can leave a VM with no network after a kernel or release upgrade.

Environment

  • Appliance: Ubuntu (seen on 24.04, and confirmed on a fresh Ubuntu 26.04 marketplace image)
  • Fresh 26.04 image: one-context 7.2.1-0, network-manager 1.54.3-2ubuntu3, kernel 7.0.0-15-generic
  • Hypervisor: KVM, single NIC

Expected behaviour

A server appliance whose NICs are managed by netplan/systemd-networkd should not also run NetworkManager. Only one network renderer should be active.

Actual state on a fresh marketplace VM

network-manager        installed (manually seeded, nothing depends on it)
NetworkManager.service active / enabled
systemd-networkd       active / enabled
/etc/netplan/          50-one-context.yaml only (renderer: networkd)
networkctl eth0        State: routable (configured)   # networkd owns it
nmcli device           eth0: unmanaged, lo: unmanaged # NM manages nothing

one-context's default on Ubuntu is NETCFG_TYPE='netplan networkd interfaces nm'. Because this is first-match-wins, netplan is always selected and the nm backend is never invoked, so one-context itself behaves correctly. The issue is that the appliance leaves the NetworkManager daemon installed and enabled alongside networkd.

Failure this triggered

We hit the consequence on a production VM (one-context 6.10.0, kernel 6.8.0-124-generic, single NIC). The VM had previously been NetworkManager-managed, so NM had written its own netplan profiles under /etc/netplan/90-NM-*.yaml (DHCP on the NIC plus a static route to a since-decommissioned network). After a kernel-upgrade reboot:

  • The VM booted to a console login prompt but was unreachable over SSH (no ARP reply, no IP on the wire).
  • systemd-networkd journal:
eth0: Link UP
eth0: Gained carrier
eth0: Could not set route: Nexthop has invalid gateway. Network is unreachable
eth0: Failed

networkd aborted the whole interface because one route (from the stale NM-rendered netplan file) had an unreachable next hop, so the one-context address was never committed. NetworkManager was at the same time trying to DHCP the same NIC.

I think the new kernel shifted renderer timing enough to turn the race into an actual outage: VM was up, networking was down, had to recover via console.

We resolved it by removing the stale 90-NM-*.yaml files and masking NetworkManager so systemd-networkd is the sole renderer. Every VM derived from these images had NM active and enabled, and several already carried the stale NM netplan files.

Impact

  • A single conflicting or stale route makes networkd fail the entire interface, not just that route.
  • It only surfaces after a reboot (typically a kernel or security upgrade), so it looks like a boot failure rather than a config conflict.
  • Recovery needs console access, which is sometimes hosed and you don't notice if you don't normally use it.

Suggested resolution / questions

  1. On Ubuntu server appliances, either do not install network-manager, or ship it disabled/masked so only systemd-networkd is active. Getting rid of it is clean, afaict nothing else depends on it.
  2. Is there a reason NetworkManager is enabled on the server appliances? If it is intended for desktop/GUI variants, can it be scoped to those images only in future? I enable NM myself of desktop-ified ON server images.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions