Skip to content

[vpp] Fix LAG subinterface hwif name resolution#1883

Merged
yue-fred-gao merged 4 commits into
sonic-net:masterfrom
Bojun-Feng:bug/lag_subinterface_hwif_name
Jun 8, 2026
Merged

[vpp] Fix LAG subinterface hwif name resolution#1883
yue-fred-gao merged 4 commits into
sonic-net:masterfrom
Bojun-Feng:bug/lag_subinterface_hwif_name

Conversation

@Bojun-Feng

Copy link
Copy Markdown
Contributor

Description of PR

LAG subinterfaces (PortChannelX.vlan) are non-functional on SONiC-VPP due to two related bugs:

  1. The RIF SUB_PORT create/delete paths call create_sub_interface(tap_to_hwif_name(dev), vlan_id, vlan_id) where dev is "PortChannelX". Since sonic_vpp_ifmap.ini only maps physical Ethernet interfaces, tap_to_hwif_name("PortChannelX") returns "Unknown" and the VPP subinterface is never created on the correct BondEthernet parent.

  2. vpp_get_hwif_name() returns early for LAG objects without appending the .vlan_id suffix, causing vpp_set_interface_mtu and vpp_set_interface_state to target the parent BondEthernet instead of the subinterface.

Changes made:

  • In the create/delete paths, check ot == SAI_OBJECT_TYPE_LAG and construct the VPP hwif name as BondEthernetX from bond_info.id instead of calling tap_to_hwif_name()
  • Fix vpp_get_hwif_name() to let the LAG branch fall through to the vlan_id append logic so MTU/state operations target the correct subinterface

Summary:
Fixes sonic-net/sonic-platform-vpp#227

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation update
  • Test improvement

Approach

What is the motivation for this PR?

LAG subinterfaces are broken on VPP platform. The test_packet_routed_with_valid_vlan[port_in_lag] test fails because the VPP subinterface is never created on the correct BondEthernet parent. Additionally, post-creation operations (MTU, admin state) would target the wrong interface due to vpp_get_hwif_name() returning the parent bond name without the .vlan_id suffix.

Work item tracking
  • Microsoft ADO (number only):

How did you do it?

Applied the same LAG-aware hwif resolution pattern already used in the VRF assignment section and in SwitchVppFdb.cpp:

  1. In the SUB_PORT create/delete paths, check ot == SAI_OBJECT_TYPE_LAG and construct "BondEthernetX" directly from bond_info.id (already populated via get_lag_bond_info()) instead of calling tap_to_hwif_name().

  2. In vpp_get_hwif_name(), restructured the LAG branch to set hwifname and fall through to the shared vlan_id append logic (the if (vlan_id) { snprintf(..., "%s.%u", ...) } block) rather than returning early with just the parent name.

How did you verify/test it?

Todo: Run test_packet_routed_with_valid_vlan[port_in_lag] in the t1-lag-vpp topology (Platform: kvm, ASIC type: vpp).

Any platform specific information?

VPP platform only. Physical port subinterfaces and non-subinterface LAG operations are unaffected (the else branches preserve original behavior).

Documentation

N/A

* Resolve PortChannel parent to BondEthernetX directly instead of calling tap_to_hwif_name() which only maps physical Ethernet interfaces
* Apply same fix to the remove path for symmetry

Signed-off-by: Bojun-Feng <bojundf@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

* Allow LAG branch to fall through to vlan_id append logic instead of returning early without the subinterface suffix

Signed-off-by: Bojun-Feng <bojundf@gmail.com>
@Bojun-Feng
Bojun-Feng force-pushed the bug/lag_subinterface_hwif_name branch from 81f62f4 to 8408572 Compare May 6, 2026 01:19
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

/azpw retry

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) jobs...

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) stages in build 1106406:

✅Stage Build:

  • Job amd64: retried.

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

/azpw retry

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) jobs...

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) stages in build 1106406:

✅Stage Build:

  • Job amd64: retried.

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

/azpw retry

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) jobs...

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) stages in build 1106406:

✅Stage Test:

  • Job vstest: retried.

✅Stage BuildTrixie:

  • Job amd64: retried.

@Bojun-Feng

Bojun-Feng commented May 6, 2026

Copy link
Copy Markdown
Contributor Author

Note to self: addRemoveDashMeterCounter and bulkChunksize unit tests are flaky due to race conditions, both tests keep failing probabilistically on different builds. I had to rerun tests multiple times.

Luckily we have the option to only rerun failed tests, else it would have taken a large number of runs to get both tests passing on all builds simutaneously.

Need to open an issue (or find existing issue) to track this.

@Bojun-Feng
Bojun-Feng marked this pull request as ready for review May 6, 2026 21:23
@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

Hi @cyw233, CI is all green. Would appreciate your review when you have time, thanks!

Additionally, I have not yet verified the fix by enabling the sonic-mgmt port_in_lag parameter for the sub-port forwarding test.

I can try to set up the environment on my end and run the test but that would take a few days. Would it be possible for you to help verify it?

@cyw233

cyw233 commented May 7, 2026

Copy link
Copy Markdown
Contributor

Hi @cyw233, CI is all green. Would appreciate your review when you have time, thanks!

Additionally, I have not yet verified the fix by enabling the sonic-mgmt port_in_lag parameter for the sub-port forwarding test.

I can try to set up the environment on my end and run the test but that would take a few days. Would it be possible for you to help verify it?

@Bojun-Feng , great to see the PR checkers are passing!

We are currently having some internal limitations/restrictions on setting up the environment and still working a solution... Maybe let's both set up our environment and run the test in parallel. Will keep you updated here

@lunyue-ms

lunyue-ms commented May 8, 2026

Copy link
Copy Markdown
Contributor

Hi @Bojun-Feng

It seems that your patch can fix the issue! Today I applied your commits and used the new image for a test. It seems that the LAG sub-interface can be displayed via vppctl. Following is the test steps.
Next week, we will verify this fix on our CI testbed in the t1-lag-vpp topology (Platform: kvm, ASIC type: vpp). Hopefully the related tests can get passed.

(BondEthernet1.10 will not be there when I test the image without your patch)

admin@sonic:~$ 
admin@sonic:~$ 
admin@sonic:~$ sudo config portchannel add PortChannel1
admin@sonic:~$ sudo config portchannel member add PortChannel1 Ethernet0
admin@sonic:~$ sudo config subinterface add PortChannel1.10 10
admin@sonic:~$ sudo config interface ip add PortChannel1.10 10.1.0.1/24
admin@sonic:~$ 
admin@sonic:~$ 
admin@sonic:~$ 
admin@sonic:~$ 
admin@sonic:~$ redis-cli -n 0 keys "*PortChannel*"
1) "LAG_MEMBER_TABLE:PortChannel1:Ethernet0"
2) "INTF_TABLE:PortChannel1.10:10.1.0.1/24"
3) "LAG_TABLE:PortChannel1"
4) "INTF_TABLE:PortChannel1.10"
admin@sonic:~$ 
admin@sonic:~$ docker exec -it syncd vppctl show interface
              Name               Idx    State  MTU (L3/IP4/IP6/MPLS)     Counter          Count     
BondEthernet1                     5      up          9000/0/0/0     tx packets                     4
                                                                    tx bytes                     520
BondEthernet1.10                  7      up          9100/0/0/0     
bobm0                             1      up          9122/0/0/0     rx packets                     4
                                                                    rx bytes                     560
                                                                    tx packets                   279
                                                                    tx bytes                   37262
                                                                    drops                          4
                                                                    ip6                            4
                                                                    rx-miss                     1135
                                                                    tx-error                       2
bobm1                             2      up          9122/0/0/0     rx packets                    24
                                                                    rx bytes                    3160
                                                                    tx packets                    45
                                                                    tx bytes                   10715
                                                                    drops                         24
                                                                    ip6                           24
                                                                    rx-miss                     1135
                                                                    tx-error                       2
local0                            0     down          0/0/0/0       drops                          2
tap4096                           3      up          9000/0/0/0     rx packets                   272
                                                                    rx bytes                   36352
                                                                    drops                          1
                                                                    ip6                           17
tap4097                           4      up          9000/0/0/0     rx packets                    46
                                                                    rx bytes                   10845
                                                                    drops                          1
                                                                    ip6                            5
tap4098                           6      up          9000/0/0/0     rx packets                     4
                                                                    rx bytes                     520
                                                                    ip6                            4
tap4098.10                        8      up           0/0/0/0       
admin@sonic:~$ 
admin@sonic:~$ show subinterfaces status
  Sub port interface    Speed    MTU    Vlan    Admin                  Type
--------------------  -------  -----  ------  -------  --------------------
     PortChannel1.10       1G   9100      10       up  802.1q-encapsulation
admin@sonic:~$ 

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

Thanks for the update @lunyue-ms! Looking forward to the CI testbed results next week. Let me know if anything comes up during the t1-lag-vpp topology run, happy to help debug if needed.

@lunyue-ms

Copy link
Copy Markdown
Contributor

Hi @Bojun-Feng

I setup my local test environment and did some tests. The VLAN related sub interface tests were failed.
test_packet_routed_with_valid_vlan.log

I'm trying to figure out what's the possible reason now and will update the ticket after.

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

Hi @Bojun-Feng

I setup my local test environment and did some tests. The VLAN related sub interface tests were failed. test_packet_routed_with_valid_vlan.log

I'm trying to figure out what's the possible reason now and will update the ticket after.

Hi @lunyue-ms, thanks for sharing! I wanted to provide some updates — I did some research on the log to look into why the [port] case (Ethernet sub-interface) is also failing, not just [port_in_lag].

Seems like the test expects ICMP replies from the DUT's sub-interfaces (e.g., Ethernet64.20, PortChannel1.20), but VPP never responds.

Looking at the code, the root cause is in SwitchVppRoute.cpp — when a connected subnet route is installed with next_hop_type = ROUTER_INTERFACE (which is the normal case for directly-connected IPs on sub-ports), the IP programming call is commented out:

if (SAI_OBJECT_TYPE_ROUTER_INTERFACE == RealObjectIdManager::objectTypeQuery(next_hop_oid))
{
// vpp_add_del_intf_ip_addr(route_entry.destination, next_hop_oid, is_add);
}

This was commented out in the upstream sonic-platform-vpp repo during the Debian Bullseye migration (sonic-net/sonic-platform-vpp#22, see here for the exact line of change).

I believe the the original vpp_add_del_intf_ip_addr function couldn't handle non-PORT interface types (LAG, loopback) and hard-rejects anything that isn't SAI_OBJECT_TYPE_PORT. The assumption was that VPP's LCP plugin would handle IP sync from Linux → VPP for all interfaces, which works for regular ports but doesn't appear to work for sub-interfaces.

There are several directions out of this:

  1. Wire the existing vpp_add_del_intf_ip_addr_norif into the ROUTER_INTERFACE case (this function already resolves interface names from prefixes and handles both PORT and LAG sub-ports).
  2. Fix the original vpp_add_del_intf_ip_addr to handle LAG/loopback types, then uncomment.
  3. Fix LCP sub-interface pairing so IP sync works implicitly (longer-term, likely a WG discussion topic).

A targeted hot-fix would be option (1) scoped to only SUB_PORT RIF types, so it doesn't risk regressing the existing LCP-based flow for regular ports. LCP improvements could be addressed separately as a longer-term item.

I think this is out of scope for this PR (which addresses LAG hwif name resolution) and would also live in a different repository. Happy to look into a follow-up hot-fix PR for this if it would be helpful.

@lunyue-ms

Copy link
Copy Markdown
Contributor

Hi @Bojun-Feng ,
Thanks for your finding! It makes sense to use the option1 in the short term. I'm looking forward to your hot-fix in another PR on this issue :)

Btw, I ran three VPP L3 sub-interface tests on the DUT (vlab-vpp-01) with tap1.20 (172.16.20.1/30) and tap2.30(172.16.30.1/30), and set up ns1/ns2 as traffic endpoints (172.16.20.2/30, 172.16.30.2/30).

  1. Test 1: ns1 -> DUT gateway (172.16.20.1)
    Failed (0/4 replies, 100% loss).
  2. Test 2: ns2 -> DUT gateway (172.16.30.1)
    Failed (0/4 replies, 100% loss).
  3. Test 3: Inter-VLAN routing through VPP (ns1 <-> ns2)
    Passed in both directions (4/4 replies each way), with TTL=63, confirming packets were routed through VPP L3 forwarding.

Additional observation: the same test logic was validated on a separate non-SONiC Ubuntu machine, and it passed there. This indicates the test method is sound, and the failure appears specific to the SONiC-VPP DUT environment (gateway/local interface reply path).
Atteched the test steps:
dut_vpp_l3_subinterface_test_report.md

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng
Bojun-Feng force-pushed the bug/lag_subinterface_hwif_name branch from 30d9e25 to b0d2aad Compare May 19, 2026 04:40
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

/azpw retry

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) jobs...

@mssonicbld

Copy link
Copy Markdown
Collaborator

Retrying failed(or canceled) stages in build 1120321:

✅Stage BuildSwss:

  • Job amd64: retried.

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

Seems like an unrelated linker error in SWSS fpmsyncd is blocking the CI. Ignoring it for now.

@lunyue-ms I have pushed some additional commits that should help with the routing_between_sub_ports and balancing_sub_ports tests.

For the tunneling_between_sub_ports tests, the tunneling feature is simply not implemented. Since its a relatively large feature (likely several hundred lines C++), I believe it deserves its own dedicated feature PR.

Please let me know if the new changes helped, thanks!

Note: The previous subport removal implementation called delete_sub_interface() followed by refresh_interfaces_list(). The new approach strips IPs and admin-downs instead, preserving the VPP sub-interface object.

The motivation was to avoid refresh_interfaces_list() during removal, which is suspected as a contributor to the test failure in routing_between_sub_ports_unaffected_by_sub_ports_removal, since it destroys and rebuilds the entire local cache. Admin-down + preserve is a lighter-weight approach that sidesteps the issue. The trade-off is that removed sub-interfaces accumulate as admin-down stubs in VPP, bounded by unique VLAN IDs per parent (max 4094).

If this approach still doesn't pass the test, the commit should be reverted. If the test is passing, then the proper long-term fix would be delete_sub_interface() + a targeted single-entry cache removal so we avoid the full rebuild.

@lunyue-ms

Copy link
Copy Markdown
Contributor

Hi @Bojun-Feng , quick status:

I opened two PRs for this work, both currently open:

Full test_sub_port_interfaces.py on vlab-vpp-01 (vms-kvm-vpp-t1-lag-vpp)
with both PRs applied: 22 passed / 10 failed. The 10 are all out of scope:

On your SVI bits (ca4ccc6f_norif): #1907 changed the SUB_PORT path to
OID-based, so picking it as-is conflicts. Happy to help rebase onto
post-#1907 master in a follow-up — with you as co-author.

@Bojun-Feng
Bojun-Feng force-pushed the bug/lag_subinterface_hwif_name branch from fb95b84 to 582a8af Compare May 25, 2026 19:53
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng

Bojun-Feng commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

Hi @lunyue-ms, thanks for the detailed update and for opening #1907! Great to see the test results improving.

I've reset this PR to the scoped LAG hwif fix only (commits 1-2). The later commits (sub-port IP programming, admin-down preservation, etc.) are superseded by your approach, so I've dropped them to avoid conflicts.

Since this PR is now a clean, self-contained fix for the LAG hwif name resolution bug (which #1907 also depends on), would it make sense to merge this one first? That way #1907 can rebase cleanly on top and we avoid duplicate commits. Happy to coordinate on timing.

For the SVI work, I'm happy to pick that up as a follow-up once #1907 lands. Thanks for the offer to collaborate on it!

@cyw233 The PR is now in its final form, just the two LAG hwif fixes you helped triage in sonic-net/sonic-platform-vpp#227. Would appreciate your review when you have a moment.

@lunyue-ms

Copy link
Copy Markdown
Contributor

Hi @Bojun-Feng, thanks for the update. I'm happy to wait for thie PR be merged and I can rebase #1907 then.

Comment thread vslib/vpp/SwitchVppRif.cpp Outdated
Signed-off-by: Bojun-Feng <bojundf@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

lunyue-ms
lunyue-ms previously approved these changes May 26, 2026

@lunyue-ms lunyue-ms left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now.

@lolyu

lolyu commented May 29, 2026

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng

Copy link
Copy Markdown
Contributor Author

Hi @cyw233 @lolyu — just checking in on this PR. CI is all green, @lunyue-ms already approved it, #1907 is waiting to rebase on top. Please let me know if there's anything else needed from my side.

@lolyu lolyu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two observations on the remove path and vpp_get_hwif_name worth addressing before merge.

Comment thread vslib/vpp/SwitchVppRif.cpp Outdated
Comment thread vslib/vpp/SwitchVppRif.cpp Outdated
* Initialize hwifname to nullptr with null-check guard in vpp_get_hwif_name
* Use BONDETHERNET_PREFIX in remove path for consistency with create path

Signed-off-by: Bojun-Feng <bojundf@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@Bojun-Feng
Bojun-Feng requested a review from lolyu June 6, 2026 16:16
@yue-fred-gao
yue-fred-gao merged commit 50de3c5 into sonic-net:master Jun 8, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PortChannel/LAG subinterfaces are not working on SONiC-VPP

6 participants