Static IPv4 on one interface can affect routing/ARP behavior of another interface in same-subnet multi-NIC setup

[xuyq19]

Problem Description

We observed that in a multi-interface setup where eth0 and eth1 are configured in the same IPv4 subnet, assigning a static IPv4 address to
eth1 may affect the routing and ARP behavior of eth0.

From the observed behavior, traffic that is expected to use eth0 can in some cases be influenced by the static IPv4 configuration on eth1,
resulting in ambiguous route selection or ARP being emitted from an unexpected interface.

Environment

• BMC system with multiple Ethernet interfaces
• Interfaces involved: eth0, eth1
• Address family: IPv4
• Network topology: eth0 and eth1 are in the same subnet
• Configuration:
  • eth0 is expected to carry management traffic
  • eth1 is configured with a static IPv4 address

Example:

• eth0: 192.168.1.10/24
• eth1: 192.168.1.20/24 static
• peer: 192.168.1.30/24

Steps to Reproduce

1. Enable both eth0 and eth1
2. Put both interfaces in the same IPv4 subnet
3. Configure a static IPv4 address on eth1
4. Keep eth0 as the interface expected to be used for management traffic
5. Access a peer in the same subnet from the BMC, or observe ARP/routing behavior

Observed Behavior

We observed one or more of the following:

• static IPv4 configuration on eth1 appears to influence overall routing behavior
• traffic expected to use eth0 may be affected by eth1
• ARP may be sent from an unexpected interface in same-subnet multi-NIC setups
• routing behavior becomes ambiguous when multiple interfaces share the same subnet

Expected Behavior

In same-subnet multi-interface setups, static IPv4 configuration on one interface should not affect another interface.

More specifically:

• static IPv4 on eth1 should not interfere with eth0
• traffic should egress through the intended interface
• ARP should be initiated from the correct interface
• route selection should remain deterministic as much as possible

Initial Analysis

Based on the current behavior, static IPv4 configuration appears to participate in routing from the perspective of the main routing table,
without stronger per-interface isolation.

In same-subnet multi-interface scenarios, this may lead to situations where:

• multiple interfaces are treated as reachable for the same prefix
• route selection lacks interface isolation
• static configuration on one interface can influence another interface

We suspect this may be related to the lack of:

• per-interface routing tables for static IPv4
• source-based policy routing rules
• dedicated handling for same-subnet multi-interface isolation

Suggestion

If this scenario is expected to be supported, it may be worth considering stronger isolation for static IPv4 in multi-interface setups, for
example:

• per-interface routing tables
• source-based policy routing
• avoiding cross-interface impact from static IPv4 configuration

If this scenario is not currently intended to be supported, it would also be helpful to clarify that in documentation.