From 1e141273c21202f80cb90ee613c03447a01989a6 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Sun, 17 May 2026 08:57:58 +0000 Subject: [PATCH 01/10] [linux-cp]: Enable lcp-sync to auto-up sub-port host netdev Sub-port host kernel netdevs (e.g. Ethernet64.20, PortChannel1.20) were created in DOWN state because VPP linux-cp plugin sets host-side link state to DOWN by default when the lcp itf-pair is created, overriding the prior 'ip link set up' that IntfMgr executed. As a result, host kernel ignored incoming sub-port traffic and PTF-driven sub-port tests (e.g. test_packet_routed_with_valid_vlan) failed because the host ICMP stack never replied to echo-request. Enable 'lcp-sync' in the linux-cp startup configuration so that VPP propagates sub-interface admin state changes to the corresponding host netdev. When sairedis brings the VPP sub-interface up via SAI API, lcp-sync now auto-ups the host counterpart, restoring the standard sub-port datapath. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-sonic-vpp/conf/startup.conf.tmpl | 1 + docker-syncd-vpp/conf/startup.conf.tmpl | 1 + 2 files changed, 2 insertions(+) diff --git a/docker-sonic-vpp/conf/startup.conf.tmpl b/docker-sonic-vpp/conf/startup.conf.tmpl index 9ba68c5..e0dfce3 100644 --- a/docker-sonic-vpp/conf/startup.conf.tmpl +++ b/docker-sonic-vpp/conf/startup.conf.tmpl @@ -302,4 +302,5 @@ plugins { # Why not support sub interfaces linux-cp { lcp-auto-subint + lcp-sync } diff --git a/docker-syncd-vpp/conf/startup.conf.tmpl b/docker-syncd-vpp/conf/startup.conf.tmpl index edb95f7..c3d2047 100644 --- a/docker-syncd-vpp/conf/startup.conf.tmpl +++ b/docker-syncd-vpp/conf/startup.conf.tmpl @@ -290,4 +290,5 @@ plugins { # Why not support sub interfaces linux-cp { lcp-auto-subint + lcp-sync } From e146d64aca2968cc470a2d3b3bb69ab2c5869180 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Mon, 18 May 2026 00:55:38 +0000 Subject: [PATCH 02/10] [dpdk]: Enable promisc on DPDK ports via dedicated supervisord program VPP DPDK virtio ports default to 'unicast off all-multicast on', which drops broadcast frames such as ARP requests targeting newly-created L3 sub-port interfaces. Real ASICs are implicitly promiscuous for routed ports, so PTF tests that rely on post-boot sub-port creation fail at the very first ARP from the peer. Manual reproduction on vlab-vpp-01: - Create Ethernet64.40 with IP 172.16.0.9/30 - From PTF eth16.40 (172.16.0.10) ping 172.16.0.9 - 0 packets arrive at bobm16 (verified via vppctl trace) - After 'vppctl set interface promiscuous on bobm16', ping succeeds VPP's startup-config directive cannot fix this because CLI commands run *before* the DPDK plugin finishes initialising its ports, so 'set interface promiscuous on bobm0' silently fails at that point. A previous attempt forked a background subshell from vpp_init.sh to wait for VPP CLI and re-apply the CLI commands. That subshell did not survive supervisord's process-group handling, so promisc was never actually enabled at runtime. Worse, the syncd container is restarted by sonic-mgmt PTF fixtures during test setup ('config reload') -- any one-shot solution loses its effect after that restart. This change replaces the fragile subshell with a dedicated supervisord program 'vpp_promisc.sh' that: 1. Waits up to 180s for the VPP CLI to come up and at least one bobm[0-9]+ interface to appear. 2. Iterates every discovered bobm* DPDK port and enables promisc. 3. Stays alive in a 2s monitor loop so promisc is re-applied after in-container VPP restarts. The new program depends on vpp_init.sh:running via supervisord_dependent_startup, mirroring the existing pattern used by syncd / rsyslogd / start. Supervisord auto-restarts the script if it exits non-zero (e.g. timeout waiting for bobm0). Applied to both docker-syncd-vpp and docker-sonic-vpp. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-sonic-vpp/Dockerfile.j2 | 2 + docker-sonic-vpp/conf/vpp_promisc.conf | 10 ++++ docker-sonic-vpp/scripts/vpp_promisc.sh | 69 +++++++++++++++++++++++++ docker-syncd-vpp/Dockerfile.j2 | 2 + docker-syncd-vpp/conf/vpp_promisc.conf | 10 ++++ docker-syncd-vpp/scripts/vpp_promisc.sh | 69 +++++++++++++++++++++++++ 6 files changed, 162 insertions(+) create mode 100644 docker-sonic-vpp/conf/vpp_promisc.conf create mode 100755 docker-sonic-vpp/scripts/vpp_promisc.sh create mode 100644 docker-syncd-vpp/conf/vpp_promisc.conf create mode 100755 docker-syncd-vpp/scripts/vpp_promisc.sh diff --git a/docker-sonic-vpp/Dockerfile.j2 b/docker-sonic-vpp/Dockerfile.j2 index adf6e75..44e2ecc 100644 --- a/docker-sonic-vpp/Dockerfile.j2 +++ b/docker-sonic-vpp/Dockerfile.j2 @@ -175,9 +175,11 @@ RUN mkdir -p /var/log/asan COPY scripts/start_sonic.sh /usr/local/bin/ COPY scripts/vpp_hostif.sh /usr/local/bin/ COPY scripts/vpp_init.sh /usr/local/bin/ +COPY scripts/vpp_promisc.sh /usr/local/bin/ COPY scripts/vppcfg_load.py /usr/local/bin/ COPY conf/vpp_init.conf /etc/supervisor/conf.d/ +COPY conf/vpp_promisc.conf /etc/supervisor/conf.d/ COPY conf/startup.conf.tmpl /etc/vpp/ COPY conf/init_cfg.json.j2 /usr/share/sonic/templates/ COPY conf/constants.yml /etc/sonic/ diff --git a/docker-sonic-vpp/conf/vpp_promisc.conf b/docker-sonic-vpp/conf/vpp_promisc.conf new file mode 100644 index 0000000..75e833b --- /dev/null +++ b/docker-sonic-vpp/conf/vpp_promisc.conf @@ -0,0 +1,10 @@ +[program:vpp_promisc.sh] +command=/usr/local/bin/vpp_promisc.sh +priority=2 +autostart=false +autorestart=true +startsecs=5 +stdout_logfile=syslog +stderr_logfile=syslog +dependent_startup=true +dependent_startup_wait_for=vpp_init.sh:running diff --git a/docker-sonic-vpp/scripts/vpp_promisc.sh b/docker-sonic-vpp/scripts/vpp_promisc.sh new file mode 100755 index 0000000..3f1da61 --- /dev/null +++ b/docker-sonic-vpp/scripts/vpp_promisc.sh @@ -0,0 +1,69 @@ +#!/bin/bash +# Copyright (c) 2026 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# vpp_promisc.sh -- keep DPDK bobm* interfaces in promiscuous mode. +# +# VPP DPDK virtio ports default to "unicast off all-multicast on", which +# drops broadcast frames (e.g. ARP requests targeting newly-created L3 +# sub-ports). Real ASICs are implicitly promiscuous for routed ports, so +# we mirror that behavior here. +# +# VPP's startup-config directive runs CLI commands before the DPDK +# plugin has finished initialising its ports, so "set interface +# promiscuous on bobm*" lines emitted into the startup config silently +# fail. This script polls until bobm* interfaces exist, then enables +# promisc; it keeps watching so the property is re-applied if VPP is +# restarted under supervisord. + +set -u + +# Optionally pull DPDK_DISABLE from the syncd env file so we can early- +# exit on DPDK-less configurations. docker-sonic-vpp may run without +# this file (env passed via "docker run -e ..."), so sourcing is best- +# effort. +VPP_ENV_FILE=/etc/sonic/vpp/syncd_vpp_env +[ -r "$VPP_ENV_FILE" ] && source "$VPP_ENV_FILE" +[ "${DPDK_DISABLE:-n}" == "y" ] && exit 0 + +WAIT_TIMEOUT=${VPP_PROMISC_WAIT:-180} +RECHECK_INTERVAL=${VPP_PROMISC_INTERVAL:-2} + +# Phase 1: wait for VPP CLI to come up and at least one bobm* interface +# to appear. Fail loudly on timeout so supervisord surfaces the problem. +deadline=$((SECONDS + WAIT_TIMEOUT)) +ready=0 +while [ $SECONDS -lt $deadline ]; do + if vppctl show interface 2>/dev/null \ + | awk '/^bobm[0-9]+ /{f=1} END{exit !f}'; then + ready=1 + break + fi + sleep 1 +done + +if [ "$ready" != 1 ]; then + echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces" >&2 + exit 1 +fi + +# Phase 2: keep promiscuous mode on for every discovered bobm* port. +# Re-apply if a new VPP instance comes up under the same supervisord. +while true; do + intfs=$(vppctl show interface 2>/dev/null \ + | awk '/^bobm[0-9]+ /{print $1}') + for intf in $intfs; do + if vppctl show hardware-interfaces "$intf" 2>/dev/null \ + | grep -q "promiscuous: unicast on"; then + continue + fi + if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then + echo "vpp_promisc.sh: enabled promisc on $intf" + fi + done + sleep "$RECHECK_INTERVAL" +done diff --git a/docker-syncd-vpp/Dockerfile.j2 b/docker-syncd-vpp/Dockerfile.j2 index 5eb25da..eb48943 100644 --- a/docker-syncd-vpp/Dockerfile.j2 +++ b/docker-syncd-vpp/Dockerfile.j2 @@ -30,8 +30,10 @@ COPY ["critical_processes", "/etc/supervisor/"] COPY scripts/vpp_hostif.sh /usr/local/bin/ COPY scripts/vpp_init.sh /usr/local/bin/ +COPY scripts/vpp_promisc.sh /usr/local/bin/ COPY conf/vpp_init.conf /etc/supervisor/conf.d/ +COPY conf/vpp_promisc.conf /etc/supervisor/conf.d/ COPY conf/startup.conf.tmpl /etc/vpp/ ## Clean up diff --git a/docker-syncd-vpp/conf/vpp_promisc.conf b/docker-syncd-vpp/conf/vpp_promisc.conf new file mode 100644 index 0000000..75e833b --- /dev/null +++ b/docker-syncd-vpp/conf/vpp_promisc.conf @@ -0,0 +1,10 @@ +[program:vpp_promisc.sh] +command=/usr/local/bin/vpp_promisc.sh +priority=2 +autostart=false +autorestart=true +startsecs=5 +stdout_logfile=syslog +stderr_logfile=syslog +dependent_startup=true +dependent_startup_wait_for=vpp_init.sh:running diff --git a/docker-syncd-vpp/scripts/vpp_promisc.sh b/docker-syncd-vpp/scripts/vpp_promisc.sh new file mode 100755 index 0000000..3f1da61 --- /dev/null +++ b/docker-syncd-vpp/scripts/vpp_promisc.sh @@ -0,0 +1,69 @@ +#!/bin/bash +# Copyright (c) 2026 Cisco and/or its affiliates. +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# vpp_promisc.sh -- keep DPDK bobm* interfaces in promiscuous mode. +# +# VPP DPDK virtio ports default to "unicast off all-multicast on", which +# drops broadcast frames (e.g. ARP requests targeting newly-created L3 +# sub-ports). Real ASICs are implicitly promiscuous for routed ports, so +# we mirror that behavior here. +# +# VPP's startup-config directive runs CLI commands before the DPDK +# plugin has finished initialising its ports, so "set interface +# promiscuous on bobm*" lines emitted into the startup config silently +# fail. This script polls until bobm* interfaces exist, then enables +# promisc; it keeps watching so the property is re-applied if VPP is +# restarted under supervisord. + +set -u + +# Optionally pull DPDK_DISABLE from the syncd env file so we can early- +# exit on DPDK-less configurations. docker-sonic-vpp may run without +# this file (env passed via "docker run -e ..."), so sourcing is best- +# effort. +VPP_ENV_FILE=/etc/sonic/vpp/syncd_vpp_env +[ -r "$VPP_ENV_FILE" ] && source "$VPP_ENV_FILE" +[ "${DPDK_DISABLE:-n}" == "y" ] && exit 0 + +WAIT_TIMEOUT=${VPP_PROMISC_WAIT:-180} +RECHECK_INTERVAL=${VPP_PROMISC_INTERVAL:-2} + +# Phase 1: wait for VPP CLI to come up and at least one bobm* interface +# to appear. Fail loudly on timeout so supervisord surfaces the problem. +deadline=$((SECONDS + WAIT_TIMEOUT)) +ready=0 +while [ $SECONDS -lt $deadline ]; do + if vppctl show interface 2>/dev/null \ + | awk '/^bobm[0-9]+ /{f=1} END{exit !f}'; then + ready=1 + break + fi + sleep 1 +done + +if [ "$ready" != 1 ]; then + echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces" >&2 + exit 1 +fi + +# Phase 2: keep promiscuous mode on for every discovered bobm* port. +# Re-apply if a new VPP instance comes up under the same supervisord. +while true; do + intfs=$(vppctl show interface 2>/dev/null \ + | awk '/^bobm[0-9]+ /{print $1}') + for intf in $intfs; do + if vppctl show hardware-interfaces "$intf" 2>/dev/null \ + | grep -q "promiscuous: unicast on"; then + continue + fi + if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then + echo "vpp_promisc.sh: enabled promisc on $intf" + fi + done + sleep "$RECHECK_INTERVAL" +done From 71799c5e2032f0ae34bd5bde3c3c3894df8f01a4 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Wed, 20 May 2026 00:02:22 +0000 Subject: [PATCH 03/10] [linux-cp]: Disable lcp-auto-subint to fix LAG sub-port LCP pair collision When VPP linux-cp lcp-auto-subint is on, VPP auto-generates host tap "be." for BondEthernet., but SONiC IntfMgr creates "PortChannel." (teamd vlan child) as the actual host netdev that holds the L3 IP. The two netdevs are different, so ARP replies land on the auto-tap, not on PortChannel., causing addNeighbor 0/8 success rate for sub-port-over-LAG. Disable lcp-auto-subint so sairedis is the sole producer of sub-port LCP pairs and can explicitly call configure_lcp_interface(BondEthernet., PortChannel.) in SwitchVpp::vpp_create_router_interface for SAI_ROUTER_INTERFACE_TYPE_SUB_PORT. Verified on vlab-vpp-01 PTF test_packet_routed_with_valid_vlan: - LCP pair table now shows itf-pair: [46] BondEthernet1.20 tap4137.20 PortChannel1.20 - No more 'configure_lcp_interface ... returned -81 EEXIST' - [port] sub-test still PASS (no regression on non-LAG sub-port) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-syncd-vpp/conf/startup.conf.tmpl | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docker-syncd-vpp/conf/startup.conf.tmpl b/docker-syncd-vpp/conf/startup.conf.tmpl index c3d2047..fc8604c 100644 --- a/docker-syncd-vpp/conf/startup.conf.tmpl +++ b/docker-syncd-vpp/conf/startup.conf.tmpl @@ -289,6 +289,12 @@ plugins { # Why not support sub interfaces linux-cp { - lcp-auto-subint + # lcp-auto-subint disabled: causes Bug 6 (LAG sub-port 0/8 addNeighbor). + # When on, VPP auto-creates host tap "be." for BondEthernet., + # but SONiC IntfMgr writes IP on "PortChannel." (teamd vlan child), + # so ARP replies go to the wrong netdev. Disable so sairedis explicitly pairs + # BondEthernet. <-> PortChannel. via configure_lcp_interface() + # in SwitchVpp::vpp_create_router_interface for SAI_ROUTER_INTERFACE_TYPE_SUB_PORT. + # lcp-auto-subint lcp-sync } From 01b7c6759ca1cb8e3aa5303a70faad5aaa2af119 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Sat, 23 May 2026 14:27:56 +0000 Subject: [PATCH 04/10] [dpdk]: Stop vpp_promisc.sh restart storm and harden promisc state check Two related hardening fixes to vpp_promisc.sh / vpp_promisc.conf applied symmetrically to docker-syncd-vpp and docker-sonic-vpp: 1. Restart storm on missing DPDK ports (phase-1 timeout). The script previously exited 1 after 180s of polling if no bobm* interface ever appeared. Combined with autorestart=true and startsecs=5, supervisord respawned the script forever: 180s wait + 'timed out' log + restart, every cycle. Two changes: - phase-1 timeout now exits 0 with a single explanatory log line ('no DPDK ports to manage') - autorestart=true -> autorestart=unexpected, so a clean exit leaves the program in EXITED state and only real crashes (segfault, bash bug) get restarted. 2. Fragile promisc state check. 'grep -q "promiscuous: unicast on"' matches an exact substring. Minor format changes between VPP releases (extra space, key reordering) would silently break the 'already on' check and cause vppctl to re-issue 'set promiscuous on' every 2s. Use a whitespace-tolerant regex 'promiscuous:[[:space:]]+unicast[[:space:]]+on' instead. No functional change for the happy path (DPDK present, VPP recent): script still polls every 2s and enables promiscuous on new bobm* ports as before. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-sonic-vpp/conf/vpp_promisc.conf | 2 +- docker-sonic-vpp/scripts/vpp_promisc.sh | 14 +++++++++++--- docker-syncd-vpp/conf/vpp_promisc.conf | 2 +- docker-syncd-vpp/scripts/vpp_promisc.sh | 14 +++++++++++--- 4 files changed, 24 insertions(+), 8 deletions(-) diff --git a/docker-sonic-vpp/conf/vpp_promisc.conf b/docker-sonic-vpp/conf/vpp_promisc.conf index 75e833b..6bbd208 100644 --- a/docker-sonic-vpp/conf/vpp_promisc.conf +++ b/docker-sonic-vpp/conf/vpp_promisc.conf @@ -2,7 +2,7 @@ command=/usr/local/bin/vpp_promisc.sh priority=2 autostart=false -autorestart=true +autorestart=unexpected startsecs=5 stdout_logfile=syslog stderr_logfile=syslog diff --git a/docker-sonic-vpp/scripts/vpp_promisc.sh b/docker-sonic-vpp/scripts/vpp_promisc.sh index 3f1da61..2ab6a65 100755 --- a/docker-sonic-vpp/scripts/vpp_promisc.sh +++ b/docker-sonic-vpp/scripts/vpp_promisc.sh @@ -47,8 +47,13 @@ while [ $SECONDS -lt $deadline ]; do done if [ "$ready" != 1 ]; then - echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces" >&2 - exit 1 + # bobm* never appeared. Most likely DPDK is not configured on this + # platform (DPDK_DISABLE may not have been set explicitly). Log once + # and exit cleanly so supervisord does not enter a restart storm. + # autorestart=unexpected (see vpp_promisc.conf) then leaves the + # program in EXITED state instead of cycling. + echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces; exiting (no DPDK ports to manage)" >&2 + exit 0 fi # Phase 2: keep promiscuous mode on for every discovered bobm* port. @@ -57,8 +62,11 @@ while true; do intfs=$(vppctl show interface 2>/dev/null \ | awk '/^bobm[0-9]+ /{print $1}') for intf in $intfs; do + # Allow trivial whitespace and format variation across VPP + # releases (e.g. extra spaces, key reordering). The previous + # fixed-substring check was fragile against minor format drift. if vppctl show hardware-interfaces "$intf" 2>/dev/null \ - | grep -q "promiscuous: unicast on"; then + | grep -qE 'promiscuous:[[:space:]]+unicast[[:space:]]+on'; then continue fi if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then diff --git a/docker-syncd-vpp/conf/vpp_promisc.conf b/docker-syncd-vpp/conf/vpp_promisc.conf index 75e833b..6bbd208 100644 --- a/docker-syncd-vpp/conf/vpp_promisc.conf +++ b/docker-syncd-vpp/conf/vpp_promisc.conf @@ -2,7 +2,7 @@ command=/usr/local/bin/vpp_promisc.sh priority=2 autostart=false -autorestart=true +autorestart=unexpected startsecs=5 stdout_logfile=syslog stderr_logfile=syslog diff --git a/docker-syncd-vpp/scripts/vpp_promisc.sh b/docker-syncd-vpp/scripts/vpp_promisc.sh index 3f1da61..2ab6a65 100755 --- a/docker-syncd-vpp/scripts/vpp_promisc.sh +++ b/docker-syncd-vpp/scripts/vpp_promisc.sh @@ -47,8 +47,13 @@ while [ $SECONDS -lt $deadline ]; do done if [ "$ready" != 1 ]; then - echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces" >&2 - exit 1 + # bobm* never appeared. Most likely DPDK is not configured on this + # platform (DPDK_DISABLE may not have been set explicitly). Log once + # and exit cleanly so supervisord does not enter a restart storm. + # autorestart=unexpected (see vpp_promisc.conf) then leaves the + # program in EXITED state instead of cycling. + echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces; exiting (no DPDK ports to manage)" >&2 + exit 0 fi # Phase 2: keep promiscuous mode on for every discovered bobm* port. @@ -57,8 +62,11 @@ while true; do intfs=$(vppctl show interface 2>/dev/null \ | awk '/^bobm[0-9]+ /{print $1}') for intf in $intfs; do + # Allow trivial whitespace and format variation across VPP + # releases (e.g. extra spaces, key reordering). The previous + # fixed-substring check was fragile against minor format drift. if vppctl show hardware-interfaces "$intf" 2>/dev/null \ - | grep -q "promiscuous: unicast on"; then + | grep -qE 'promiscuous:[[:space:]]+unicast[[:space:]]+on'; then continue fi if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then From cbfb6ed19ff189775b342538c8d51fc6d3b55d65 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Mon, 25 May 2026 12:55:47 +0000 Subject: [PATCH 05/10] [docs/HLD]: Document LAG sub-port architecture Document the LAG sub-port (PortChannel.) plumbing introduced in sonic-sairedis PR #1907 and the matching platform-vpp changes (startup.conf 'lcp-auto-subint = disabled', vpp_promisc.sh monitor loop). vpp-lag.md: - TOC, Revisions (v0.3) and Scope: add 'LAG Subinterface' entries. - New section 'LAG Subinterface' explaining the 3-tuple naming (PortChannel. / BondEthernet. / be.), the bidirectional 'tc mirred' bridge between LCP host and SONiC L3 netdev, why 'lcp-auto-subint=disabled' is required, and the RIF IP / ARP handling (idempotent -105, arp_accept=1, neighbor pre-warm). - Status row updated to reflect 'Hwif naming + explicit LCP pair + tc bridge + RIF IP idempotency + arp_accept/pre-warm + basic [port_in_lag] PTF PASS', with a follow-up note for DUT-originated ARP on a few PTF cases (root cause to be diagnosed). SONICVPP-HLD.md: - Subinterface row: align with current behaviour (explicit configure_lcp_interface from sairedis with 'lcp-auto-subint=disabled'). - New LAG Subinterface row covering BondEthernet. + be. LCP pair and the bidirectional tc bridge to PortChannel.. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docs/HLD/SONICVPP-HLD.md | 3 ++- docs/HLD/vpp-lag.md | 37 +++++++++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/docs/HLD/SONICVPP-HLD.md b/docs/HLD/SONICVPP-HLD.md index c5eac98..c9f5ab7 100644 --- a/docs/HLD/SONICVPP-HLD.md +++ b/docs/HLD/SONICVPP-HLD.md @@ -360,7 +360,8 @@ The following table summarizes the flow from configuration to VPP/LCP API for di | **Front-Panel** | Loaded from `config_db.json` | `\|c\|SAI_OBJECT_TYPE_HOSTIF:oid:0xd000000000080`
`SAI_HOSTIF_ATTR_NAME=Ethernet0`| *vpp intf pre-created using generated `startup.conf` | `lcp_itf_pair_add_del(vpp_ifname,EthernetX)`

>`itf-pair: [0] bobm0 tap4096 Ethernet0 10 type tap` | N/A | | **PortChannel** | sudo config portchannel add PortChannel10
sudo config portchannel member add PortChannel10 Ethernet4 | `\|c\|SAI_OBJECT_TYPE_LAG:oid:0x2000000000095`
`\|c\|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000096`
`SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x2000000000095` | `bond_create`
`bond_add_member` | `lcp_itf_pair_add_del(BondEthernetX,beX)`

>`itf-pair: [4] BondEthernet10 tap4099 be10 16 type tap` | `beX`
-->
`PortChannelX` | | **Loopback** | sudo config interface ip add Loopback0 10.0.0.1/32 | `\|c\|SAI_OBJECT_TYPE_ROUTE_ENTRY:{"dest":"10.0.0.1/32","` | `create_loopback_instance`
`sw_interface_add_del_address` | `lcp_itf_pair_add_del(loopX,tap_LoopbackX)`

>`itf-pair: [2] loop0 tap4098 tap_Loopback0 13 type tap` | `tap_LoopbackX`
-->
`LoopbackX` | -| **Subinterface** | sudo config subinterface add Ethernet0.10 10 | `\|c\|SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000094`
`SAI_ROUTER_INTERFACE_ATTR_TYPE=SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` | `create_subif`| `lcp_itf_pair_add_del` automatically invoked with `lcp-auto-subinf` enabled

>`itf-pair: [3] bobm0.10 tap4096.10 Ethernet0.10 14 type tap` | N/A, shares parent tap | +| **Subinterface** | sudo config subinterface add Ethernet0.10 10 | `\|c\|SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000094`
`SAI_ROUTER_INTERFACE_ATTR_TYPE=SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` | `create_subif`| Explicit `configure_lcp_interface(bobmX.10, EthernetX.10)` from sairedis (with `lcp-auto-subint = disabled` in startup.conf)

>`itf-pair: [3] bobm0.10 tap4096.10 Ethernet0.10 14 type tap` | N/A, shares parent tap | +| **LAG Subinterface** | sudo config subinterface add PortChannel10.20 20 | `\|c\|SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000095`
`SAI_ROUTER_INTERFACE_ATTR_TYPE=SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` (parent is a LAG) | `create_subif` on `BondEthernet10`, named `BondEthernet10.20` | Explicit `configure_lcp_interface(BondEthernet10.20, be10.20)` from sairedis (the SONiC L3 netdev `PortChannel10.20` is a teamd VLAN child, not a VPP tap, so cannot be the LCP host directly)

>`itf-pair: [N] BondEthernet10.20 tap4099.20 be10.20 N type tap` | `be10.20`
<-->
`PortChannel10.20`
(bidirectional `tc mirred` so ARP/L3 traffic reaches the SONiC L3 netdev that holds the IP) | diff --git a/docs/HLD/vpp-lag.md b/docs/HLD/vpp-lag.md index b6e2183..cbb1b13 100644 --- a/docs/HLD/vpp-lag.md +++ b/docs/HLD/vpp-lag.md @@ -13,7 +13,8 @@ Rev v0.2 5. [LAG and Bridging](#item-5) 6. [LACP Support](#item-6) 7. [LAG L3 Support](#item-7) -8. [Status](#item-8) +8. [LAG Subinterface](#item-8) +9. [Status](#item-9)

@@ -25,6 +26,7 @@ Rev v0.2 |-----|------|-----------|---------| |v0.1 | 25/02/2024 | Bendrapu Balareddy (Cisco), Sameer Nanajkar (Cisco) | LAG & Bridging Support | |v0.2 | 17/12/2024 | Akeel Ali (Cisco) | LACP & L3 Support | +|v0.3 | 25/05/2026 | Bojun Feng, Lun Yue (Microsoft) | LAG Subinterface — basic sub-port path |
@@ -35,6 +37,8 @@ Rev v0.2 This document describes the high level design of integrating LAG between SONIC and VPP. It covers: - LAG and 802.1Q bridging - LACP + - LAG L3 (PortChannel with IP) + - LAG Subinterface (PortChannel\.\)
@@ -132,6 +136,35 @@ tc filter add dev parent ffff: \ +## LAG Subinterface + +PortChannel subinterfaces are tagged L3 interfaces on top of a LAG (`PortChannel.`), used for example by the t1-lag topology where every LAG carries multiple `/30` peering subnets, one per VLAN. + +A LAG subinterface has three coupled names that must stay consistent: `PortChannel.` (the SONiC L3 kernel netdev that teamd creates and that the IP lives on), `BondEthernet.` (the VPP sub-interface on the bond), and `be.` (the LCP host, a VLAN child of the dummy tap `be`). On `SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` create, `vpp_get_hwif_name` must return `BondEthernet.`; returning the parent bond name causes every subsequent SAI call on the sub-interface (IP add, neighbor add, delete) to operate on the wrong VPP object. + +### Plumbing + +For a non-LAG sub-port (`Ethernet0.10`) the LCP host can simply be the SONiC L3 netdev itself, because it sits on top of the VPP-created tap `tap4096`. That does not work for a LAG sub-port: `PortChannel.` is a teamd VLAN child, not a VPP tap, and VPP linux-cp can only punt to a kernel netdev backed by a VPP tap. + +The design pairs the VPP sub-interface with `be.` (a VLAN child of the dummy tap `be` that already exists for the parent LAG), and then bridges that LCP host to the SONiC L3 netdev `PortChannel.` with a bidirectional `tc mirred` redirect. The ingress redirect (`be.` → `PortChannel.`) carries VPP-punted ARP/ICMP frames to the netdev that holds the IP, so the kernel handles them normally. The egress redirect (`PortChannel.` → `be.`) carries kernel-originated frames into VPP via the LCP tap, where the linux-cp cross-connect forwards them to `BondEthernet.`. Both directions are mandatory: a one-direction-only bridge silently breaks LAG sub-port ping, and failure of either direction rolls back the LCP pair and VPP sub-interface so the SAI RIF create fails cleanly. + +`lcp-auto-subint` is disabled in `startup.conf.tmpl` so sairedis owns sub-port LCP pair creation end-to-end — the pair and the tc bridge go in together as part of one SAI RIF create. Leaving it enabled lets VPP create the pair on `create_subif` before the tc bridge is in place, leaving ARP replies VPP punts to `be.` with no path to `PortChannel.` (where the IP lives); the kernel ARP table on `PortChannel.` stays empty and `addNeighbor` for sub-port-over-LAG fails (the original "Bug 6: LAG sub-port 0/8 addNeighbor"). + +### RIF IP and ARP + +The RIF IP is programmed by the same `vpp_add_del_intf_ip_addr_norif` flow used for PortChannel L3 ([Configuring IP](#item-7)), extended to recognise `BondEthernet.` names so the IP lands on the sub-interface rather than on the parent bond. VPP returns `ADDRESS_IN_USE (-105)` when orchagent re-issues the same IP add on `config reload` or on transient connected-route remove/add cycles; the handler treats `-105` as success on add, and the transient remove path no longer clears VPP's RIF IP, so a single `config reload` does not permanently lose sub-interface IP state. + +The kernel ignores unsolicited ARP replies on a sub-port netdev by default (`arp_accept = 0`), which would otherwise break the VPP → kernel population path on the LCP host. `vslib/vpp` sets `net.ipv4.conf..arp_accept = 1` on RIF create, and pre-warms the VPP neighbor table on RIF IP add so the first packet does not have to wait for resolution. + +### Known limitations + +DUT-originated ARP on a LAG sub-port can still fail to install a neighbor into the VPP sub-interface RIF in a small number of PTF cases that depend on DUT-initiated traffic (i.e. where no prior ingress packet seeds the kernel ARP table). Non-LAG (`port`) sub-port equivalents do not exhibit the same failure pattern, which suggests the issue lies in the LAG-specific plumbing — most likely somewhere along the bidirectional `tc mirred` bridge or the kernel → VPP propagation that sits on top of it. A diagnostic pass and targeted fix will be tracked in a follow-up PR. + +
+
+ + + ## Status @@ -143,7 +176,7 @@ Ping between PortChannels | Coded solution using `tc` utility to redirect traffi Testing | Bring-up, ping of PortChannel with 2 members, add/remove members, v4/v6/multiple-members, multiple-portchannels, L2 traffic | Sonic-mgmt t1-28-lag testing | Brought-up t1-28-lag TOPO. Ran LAG tests |
  • Debug and fix failing LAG TCs
    • Hashing algo selection | Coded using XOR & L3L4 | (Optional, TBD) Ability to switchover between L2L3 and L3L4 depending on presence of IP (or other scheme) -(Phase 2 - T0-lag) PortChannel Subinterfaces | Identified changes required to provision PortChannel subintf in VPP and apply IP |
  • Ping fails
  • May require redesign to add PortChannel support to existing interface APIs
  • Bring-up t0-lag topo and run LAG tests
    • +(T0-lag) PortChannel Subinterfaces |
  • Hwif naming `BondEthernet.`
  • Explicit LCP pair `BondEthernet.` ↔ `be.` plus bidirectional `tc mirred` bridge `be.` ↔ `PortChannel.` (with `lcp-auto-subint=disabled` so sairedis is the sole producer of the pair)
  • RIF IP programming on VPP sub-interface, idempotent on `-105`, preserved across transient remove cycles
  • `arp_accept=1` and VPP neighbor pre-warm on sub-port RIF create
  • PTF `t1-lag-vpp`: basic `[port_in_lag]` sub-port routing PASS
    • |
  • DUT-originated ARP on LAG sub-port can still fail to install reply in sub-interface RIF — affects a few cross-port / balancing PTF cases; follow-up to diagnose where in the tc bridge / kernel → VPP propagation the ARP is lost
  • SVI variants (sonic-buildimage #26936) and IPinIP tunneling (sairedis #1860) — separate features
    • From dfb29025eebec1721dd72f87fbc0a5ea714410bf Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Tue, 26 May 2026 01:09:26 +0000 Subject: [PATCH 06/10] [linux-cp]: Align lcp-auto-subint state across syncd-vpp and sonic-vpp Both docker-syncd-vpp and docker-sonic-vpp now disable lcp-auto-subint and carry the same rationale comment, so sairedis is the sole producer of LAG sub-port LCP pairs in either deployment model (multi-container VM image or single-container sonic-vpp image). Also rewrites the existing rationale comment in docker-syncd-vpp to be more accurate: configure_lcp_interface pairs BondEthernet. with be. (not with PortChannel.); the bridge to PortChannel. is a separate tc mirred step in the same SAI RIF create. ARP replies do reach the correct LCP host tap; the kernel drops them because the IP lives on a different netdev. Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docker-sonic-vpp/conf/startup.conf.tmpl | 8 +++++++- docker-syncd-vpp/conf/startup.conf.tmpl | 12 ++++++------ 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/docker-sonic-vpp/conf/startup.conf.tmpl b/docker-sonic-vpp/conf/startup.conf.tmpl index e0dfce3..6948cc5 100644 --- a/docker-sonic-vpp/conf/startup.conf.tmpl +++ b/docker-sonic-vpp/conf/startup.conf.tmpl @@ -301,6 +301,12 @@ plugins { # Why not support sub interfaces linux-cp { - lcp-auto-subint + # lcp-auto-subint disabled: auto-pair "BondEthernet." <-> "be." + # punts ARP to be., but SONiC writes the sub-port IP on + # "PortChannel." (separate teamd vlan netdev), so the kernel drops + # the unsolicited ARP and the VPP neighbor table never gets populated. + # Disabled so sairedis (vpp_create_router_interface, SUB_PORT path) creates the + # LCP pair + tc mirred bridge between be. and PortChannel.. + # lcp-auto-subint lcp-sync } diff --git a/docker-syncd-vpp/conf/startup.conf.tmpl b/docker-syncd-vpp/conf/startup.conf.tmpl index fc8604c..a6991d9 100644 --- a/docker-syncd-vpp/conf/startup.conf.tmpl +++ b/docker-syncd-vpp/conf/startup.conf.tmpl @@ -289,12 +289,12 @@ plugins { # Why not support sub interfaces linux-cp { - # lcp-auto-subint disabled: causes Bug 6 (LAG sub-port 0/8 addNeighbor). - # When on, VPP auto-creates host tap "be." for BondEthernet., - # but SONiC IntfMgr writes IP on "PortChannel." (teamd vlan child), - # so ARP replies go to the wrong netdev. Disable so sairedis explicitly pairs - # BondEthernet. <-> PortChannel. via configure_lcp_interface() - # in SwitchVpp::vpp_create_router_interface for SAI_ROUTER_INTERFACE_TYPE_SUB_PORT. + # lcp-auto-subint disabled: auto-pair "BondEthernet." <-> "be." + # punts ARP to be., but SONiC writes the sub-port IP on + # "PortChannel." (separate teamd vlan netdev), so the kernel drops + # the unsolicited ARP and the VPP neighbor table never gets populated. + # Disabled so sairedis (vpp_create_router_interface, SUB_PORT path) creates the + # LCP pair + tc mirred bridge between be. and PortChannel.. # lcp-auto-subint lcp-sync } From 30359dbda771232ca47c8a0b1d45170e1a445e80 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Sun, 31 May 2026 09:46:48 +0000 Subject: [PATCH 07/10] Remove vpp_promisc.sh; promisc now set via sairedis VPP API Promiscuous mode on the VPP DPDK physical interfaces is now configured directly through the sairedis VPP API (interface_set_promiscuous, called at LCP pair creation and after bond member enslave) instead of the supervisord-managed vpp_promisc.sh polling script. Remove the now-redundant vpp_promisc.sh / vpp_promisc.conf and their COPY lines from both docker-sonic-vpp and docker-syncd-vpp. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-sonic-vpp/Dockerfile.j2 | 2 - docker-sonic-vpp/conf/vpp_promisc.conf | 10 ---- docker-sonic-vpp/scripts/vpp_promisc.sh | 77 ------------------------- docker-syncd-vpp/Dockerfile.j2 | 2 - docker-syncd-vpp/conf/vpp_promisc.conf | 10 ---- docker-syncd-vpp/scripts/vpp_promisc.sh | 77 ------------------------- 6 files changed, 178 deletions(-) delete mode 100644 docker-sonic-vpp/conf/vpp_promisc.conf delete mode 100755 docker-sonic-vpp/scripts/vpp_promisc.sh delete mode 100644 docker-syncd-vpp/conf/vpp_promisc.conf delete mode 100755 docker-syncd-vpp/scripts/vpp_promisc.sh diff --git a/docker-sonic-vpp/Dockerfile.j2 b/docker-sonic-vpp/Dockerfile.j2 index 44e2ecc..adf6e75 100644 --- a/docker-sonic-vpp/Dockerfile.j2 +++ b/docker-sonic-vpp/Dockerfile.j2 @@ -175,11 +175,9 @@ RUN mkdir -p /var/log/asan COPY scripts/start_sonic.sh /usr/local/bin/ COPY scripts/vpp_hostif.sh /usr/local/bin/ COPY scripts/vpp_init.sh /usr/local/bin/ -COPY scripts/vpp_promisc.sh /usr/local/bin/ COPY scripts/vppcfg_load.py /usr/local/bin/ COPY conf/vpp_init.conf /etc/supervisor/conf.d/ -COPY conf/vpp_promisc.conf /etc/supervisor/conf.d/ COPY conf/startup.conf.tmpl /etc/vpp/ COPY conf/init_cfg.json.j2 /usr/share/sonic/templates/ COPY conf/constants.yml /etc/sonic/ diff --git a/docker-sonic-vpp/conf/vpp_promisc.conf b/docker-sonic-vpp/conf/vpp_promisc.conf deleted file mode 100644 index 6bbd208..0000000 --- a/docker-sonic-vpp/conf/vpp_promisc.conf +++ /dev/null @@ -1,10 +0,0 @@ -[program:vpp_promisc.sh] -command=/usr/local/bin/vpp_promisc.sh -priority=2 -autostart=false -autorestart=unexpected -startsecs=5 -stdout_logfile=syslog -stderr_logfile=syslog -dependent_startup=true -dependent_startup_wait_for=vpp_init.sh:running diff --git a/docker-sonic-vpp/scripts/vpp_promisc.sh b/docker-sonic-vpp/scripts/vpp_promisc.sh deleted file mode 100755 index 2ab6a65..0000000 --- a/docker-sonic-vpp/scripts/vpp_promisc.sh +++ /dev/null @@ -1,77 +0,0 @@ -#!/bin/bash -# Copyright (c) 2026 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# vpp_promisc.sh -- keep DPDK bobm* interfaces in promiscuous mode. -# -# VPP DPDK virtio ports default to "unicast off all-multicast on", which -# drops broadcast frames (e.g. ARP requests targeting newly-created L3 -# sub-ports). Real ASICs are implicitly promiscuous for routed ports, so -# we mirror that behavior here. -# -# VPP's startup-config directive runs CLI commands before the DPDK -# plugin has finished initialising its ports, so "set interface -# promiscuous on bobm*" lines emitted into the startup config silently -# fail. This script polls until bobm* interfaces exist, then enables -# promisc; it keeps watching so the property is re-applied if VPP is -# restarted under supervisord. - -set -u - -# Optionally pull DPDK_DISABLE from the syncd env file so we can early- -# exit on DPDK-less configurations. docker-sonic-vpp may run without -# this file (env passed via "docker run -e ..."), so sourcing is best- -# effort. -VPP_ENV_FILE=/etc/sonic/vpp/syncd_vpp_env -[ -r "$VPP_ENV_FILE" ] && source "$VPP_ENV_FILE" -[ "${DPDK_DISABLE:-n}" == "y" ] && exit 0 - -WAIT_TIMEOUT=${VPP_PROMISC_WAIT:-180} -RECHECK_INTERVAL=${VPP_PROMISC_INTERVAL:-2} - -# Phase 1: wait for VPP CLI to come up and at least one bobm* interface -# to appear. Fail loudly on timeout so supervisord surfaces the problem. -deadline=$((SECONDS + WAIT_TIMEOUT)) -ready=0 -while [ $SECONDS -lt $deadline ]; do - if vppctl show interface 2>/dev/null \ - | awk '/^bobm[0-9]+ /{f=1} END{exit !f}'; then - ready=1 - break - fi - sleep 1 -done - -if [ "$ready" != 1 ]; then - # bobm* never appeared. Most likely DPDK is not configured on this - # platform (DPDK_DISABLE may not have been set explicitly). Log once - # and exit cleanly so supervisord does not enter a restart storm. - # autorestart=unexpected (see vpp_promisc.conf) then leaves the - # program in EXITED state instead of cycling. - echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces; exiting (no DPDK ports to manage)" >&2 - exit 0 -fi - -# Phase 2: keep promiscuous mode on for every discovered bobm* port. -# Re-apply if a new VPP instance comes up under the same supervisord. -while true; do - intfs=$(vppctl show interface 2>/dev/null \ - | awk '/^bobm[0-9]+ /{print $1}') - for intf in $intfs; do - # Allow trivial whitespace and format variation across VPP - # releases (e.g. extra spaces, key reordering). The previous - # fixed-substring check was fragile against minor format drift. - if vppctl show hardware-interfaces "$intf" 2>/dev/null \ - | grep -qE 'promiscuous:[[:space:]]+unicast[[:space:]]+on'; then - continue - fi - if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then - echo "vpp_promisc.sh: enabled promisc on $intf" - fi - done - sleep "$RECHECK_INTERVAL" -done diff --git a/docker-syncd-vpp/Dockerfile.j2 b/docker-syncd-vpp/Dockerfile.j2 index eb48943..5eb25da 100644 --- a/docker-syncd-vpp/Dockerfile.j2 +++ b/docker-syncd-vpp/Dockerfile.j2 @@ -30,10 +30,8 @@ COPY ["critical_processes", "/etc/supervisor/"] COPY scripts/vpp_hostif.sh /usr/local/bin/ COPY scripts/vpp_init.sh /usr/local/bin/ -COPY scripts/vpp_promisc.sh /usr/local/bin/ COPY conf/vpp_init.conf /etc/supervisor/conf.d/ -COPY conf/vpp_promisc.conf /etc/supervisor/conf.d/ COPY conf/startup.conf.tmpl /etc/vpp/ ## Clean up diff --git a/docker-syncd-vpp/conf/vpp_promisc.conf b/docker-syncd-vpp/conf/vpp_promisc.conf deleted file mode 100644 index 6bbd208..0000000 --- a/docker-syncd-vpp/conf/vpp_promisc.conf +++ /dev/null @@ -1,10 +0,0 @@ -[program:vpp_promisc.sh] -command=/usr/local/bin/vpp_promisc.sh -priority=2 -autostart=false -autorestart=unexpected -startsecs=5 -stdout_logfile=syslog -stderr_logfile=syslog -dependent_startup=true -dependent_startup_wait_for=vpp_init.sh:running diff --git a/docker-syncd-vpp/scripts/vpp_promisc.sh b/docker-syncd-vpp/scripts/vpp_promisc.sh deleted file mode 100755 index 2ab6a65..0000000 --- a/docker-syncd-vpp/scripts/vpp_promisc.sh +++ /dev/null @@ -1,77 +0,0 @@ -#!/bin/bash -# Copyright (c) 2026 Cisco and/or its affiliates. -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at: -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# vpp_promisc.sh -- keep DPDK bobm* interfaces in promiscuous mode. -# -# VPP DPDK virtio ports default to "unicast off all-multicast on", which -# drops broadcast frames (e.g. ARP requests targeting newly-created L3 -# sub-ports). Real ASICs are implicitly promiscuous for routed ports, so -# we mirror that behavior here. -# -# VPP's startup-config directive runs CLI commands before the DPDK -# plugin has finished initialising its ports, so "set interface -# promiscuous on bobm*" lines emitted into the startup config silently -# fail. This script polls until bobm* interfaces exist, then enables -# promisc; it keeps watching so the property is re-applied if VPP is -# restarted under supervisord. - -set -u - -# Optionally pull DPDK_DISABLE from the syncd env file so we can early- -# exit on DPDK-less configurations. docker-sonic-vpp may run without -# this file (env passed via "docker run -e ..."), so sourcing is best- -# effort. -VPP_ENV_FILE=/etc/sonic/vpp/syncd_vpp_env -[ -r "$VPP_ENV_FILE" ] && source "$VPP_ENV_FILE" -[ "${DPDK_DISABLE:-n}" == "y" ] && exit 0 - -WAIT_TIMEOUT=${VPP_PROMISC_WAIT:-180} -RECHECK_INTERVAL=${VPP_PROMISC_INTERVAL:-2} - -# Phase 1: wait for VPP CLI to come up and at least one bobm* interface -# to appear. Fail loudly on timeout so supervisord surfaces the problem. -deadline=$((SECONDS + WAIT_TIMEOUT)) -ready=0 -while [ $SECONDS -lt $deadline ]; do - if vppctl show interface 2>/dev/null \ - | awk '/^bobm[0-9]+ /{f=1} END{exit !f}'; then - ready=1 - break - fi - sleep 1 -done - -if [ "$ready" != 1 ]; then - # bobm* never appeared. Most likely DPDK is not configured on this - # platform (DPDK_DISABLE may not have been set explicitly). Log once - # and exit cleanly so supervisord does not enter a restart storm. - # autorestart=unexpected (see vpp_promisc.conf) then leaves the - # program in EXITED state instead of cycling. - echo "vpp_promisc.sh: timed out after ${WAIT_TIMEOUT}s waiting for bobm* interfaces; exiting (no DPDK ports to manage)" >&2 - exit 0 -fi - -# Phase 2: keep promiscuous mode on for every discovered bobm* port. -# Re-apply if a new VPP instance comes up under the same supervisord. -while true; do - intfs=$(vppctl show interface 2>/dev/null \ - | awk '/^bobm[0-9]+ /{print $1}') - for intf in $intfs; do - # Allow trivial whitespace and format variation across VPP - # releases (e.g. extra spaces, key reordering). The previous - # fixed-substring check was fragile against minor format drift. - if vppctl show hardware-interfaces "$intf" 2>/dev/null \ - | grep -qE 'promiscuous:[[:space:]]+unicast[[:space:]]+on'; then - continue - fi - if vppctl set interface promiscuous on "$intf" >/dev/null 2>&1; then - echo "vpp_promisc.sh: enabled promisc on $intf" - fi - done - sleep "$RECHECK_INTERVAL" -done From 37a4d40bee3a35d9fd28622ca79c5021af3552db Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Tue, 2 Jun 2026 09:45:11 +0000 Subject: [PATCH 08/10] [docs/HLD]: Align LAG subinterface HLD with implementation Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docs/HLD/vpp-lag.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/docs/HLD/vpp-lag.md b/docs/HLD/vpp-lag.md index cbb1b13..3402e9e 100644 --- a/docs/HLD/vpp-lag.md +++ b/docs/HLD/vpp-lag.md @@ -144,7 +144,7 @@ A LAG subinterface has three coupled names that must stay consistent: `PortChann ### Plumbing -For a non-LAG sub-port (`Ethernet0.10`) the LCP host can simply be the SONiC L3 netdev itself, because it sits on top of the VPP-created tap `tap4096`. That does not work for a LAG sub-port: `PortChannel.` is a teamd VLAN child, not a VPP tap, and VPP linux-cp can only punt to a kernel netdev backed by a VPP tap. +For a non-LAG sub-port (`Ethernet0.10`) the LCP host can simply be the SONiC L3 netdev itself, because it sits on top of a VPP-created tap (`tap`). That does not work for a LAG sub-port: `PortChannel.` is a teamd VLAN child, not a VPP tap, and VPP linux-cp can only punt to a kernel netdev backed by a VPP tap. The design pairs the VPP sub-interface with `be.` (a VLAN child of the dummy tap `be` that already exists for the parent LAG), and then bridges that LCP host to the SONiC L3 netdev `PortChannel.` with a bidirectional `tc mirred` redirect. The ingress redirect (`be.` → `PortChannel.`) carries VPP-punted ARP/ICMP frames to the netdev that holds the IP, so the kernel handles them normally. The egress redirect (`PortChannel.` → `be.`) carries kernel-originated frames into VPP via the LCP tap, where the linux-cp cross-connect forwards them to `BondEthernet.`. Both directions are mandatory: a one-direction-only bridge silently breaks LAG sub-port ping, and failure of either direction rolls back the LCP pair and VPP sub-interface so the SAI RIF create fails cleanly. @@ -152,14 +152,10 @@ The design pairs the VPP sub-interface with `be.` (a VLAN child of the ### RIF IP and ARP -The RIF IP is programmed by the same `vpp_add_del_intf_ip_addr_norif` flow used for PortChannel L3 ([Configuring IP](#item-7)), extended to recognise `BondEthernet.` names so the IP lands on the sub-interface rather than on the parent bond. VPP returns `ADDRESS_IN_USE (-105)` when orchagent re-issues the same IP add on `config reload` or on transient connected-route remove/add cycles; the handler treats `-105` as success on add, and the transient remove path no longer clears VPP's RIF IP, so a single `config reload` does not permanently lose sub-interface IP state. +Sub-port connected route programming can arrive through both `ROUTER_INTERFACE` and `PORT` next-hop route events. The `ROUTER_INTERFACE` / `SUB_PORT` branch uses `vpp_add_del_intf_ip_addr()` to resolve the RIF directly from the SAI RIF OID, while the `PORT` next-hop branch keeps using `vpp_add_del_intf_ip_addr_norif()` and resolves the netdev by prefix from the host kernel. Both paths resolve LAG sub-ports to `BondEthernet.` so the IP lands on the sub-interface rather than on the parent bond. VPP returns `ADDRESS_IN_USE (-105)` when orchagent re-issues the same IP add on `config reload` or on transient connected-route remove/add cycles; the handler treats `-105` as success on add, and the transient remove path no longer clears VPP's RIF IP, so a single `config reload` does not permanently lose sub-interface IP state. The kernel ignores unsolicited ARP replies on a sub-port netdev by default (`arp_accept = 0`), which would otherwise break the VPP → kernel population path on the LCP host. `vslib/vpp` sets `net.ipv4.conf..arp_accept = 1` on RIF create, and pre-warms the VPP neighbor table on RIF IP add so the first packet does not have to wait for resolution. -### Known limitations - -DUT-originated ARP on a LAG sub-port can still fail to install a neighbor into the VPP sub-interface RIF in a small number of PTF cases that depend on DUT-initiated traffic (i.e. where no prior ingress packet seeds the kernel ARP table). Non-LAG (`port`) sub-port equivalents do not exhibit the same failure pattern, which suggests the issue lies in the LAG-specific plumbing — most likely somewhere along the bidirectional `tc mirred` bridge or the kernel → VPP propagation that sits on top of it. A diagnostic pass and targeted fix will be tracked in a follow-up PR. -

    @@ -176,7 +172,7 @@ Ping between PortChannels | Coded solution using `tc` utility to redirect traffi Testing | Bring-up, ping of PortChannel with 2 members, add/remove members, v4/v6/multiple-members, multiple-portchannels, L2 traffic | Sonic-mgmt t1-28-lag testing | Brought-up t1-28-lag TOPO. Ran LAG tests |
  • Debug and fix failing LAG TCs
    • Hashing algo selection | Coded using XOR & L3L4 | (Optional, TBD) Ability to switchover between L2L3 and L3L4 depending on presence of IP (or other scheme) -(T0-lag) PortChannel Subinterfaces |
  • Hwif naming `BondEthernet.`
  • Explicit LCP pair `BondEthernet.` ↔ `be.` plus bidirectional `tc mirred` bridge `be.` ↔ `PortChannel.` (with `lcp-auto-subint=disabled` so sairedis is the sole producer of the pair)
  • RIF IP programming on VPP sub-interface, idempotent on `-105`, preserved across transient remove cycles
  • `arp_accept=1` and VPP neighbor pre-warm on sub-port RIF create
  • PTF `t1-lag-vpp`: basic `[port_in_lag]` sub-port routing PASS
    • |
  • DUT-originated ARP on LAG sub-port can still fail to install reply in sub-interface RIF — affects a few cross-port / balancing PTF cases; follow-up to diagnose where in the tc bridge / kernel → VPP propagation the ARP is lost
  • SVI variants (sonic-buildimage #26936) and IPinIP tunneling (sairedis #1860) — separate features
    • +(T0-lag) PortChannel Subinterfaces |
  • Hwif naming `BondEthernet.`
  • Explicit LCP pair `BondEthernet.` ↔ `be.` plus bidirectional `tc mirred` bridge `be.` ↔ `PortChannel.` (with `lcp-auto-subint=disabled` so sairedis is the sole producer of the pair)
  • RIF IP programming on VPP sub-interface, idempotent on `-105`, preserved across transient remove cycles
  • `arp_accept=1` and VPP neighbor pre-warm on sub-port RIF create
  • Representative `t1-lag-vpp` sub-port smoke tests PASS
    • |
  • SVI variants (sonic-buildimage #26936) and IPinIP tunneling (sairedis #1860) — separate features
    • @@ -184,4 +180,3 @@ Hashing algo selection | Coded using XOR & L3L4 | (Optional, TBD) Ability to swi [SONiC system architecture](https://github.com/sonic-net/SONiC/wiki/Architecture)\ [What is VPP](https://s3-docs.fd.io/vpp/23.06/) - From 46f448c50f11a0ad521207714d150e51e00978ad Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Mon, 15 Jun 2026 02:51:49 +0000 Subject: [PATCH 09/10] [docs/HLD]: revert LAG sub-port HLD; keep plain Ethernet sub-port LAG sub-port host/CP punt is being redesigned (a dedicated VPP direct-TX node, per review on this PR). The data plane is non-functional in the meantime and the LAG sub-port forwarding tests are skipped in sonic-mgmt, so documenting the (now-removed) tc-mirred punt design here is premature. - vpp-lag.md: revert entirely to its pre-PR state. Every addition this PR made to it (the "LAG Subinterface" section, TOC/Revisions/Intro entries and the Status row rewrite) was LAG-sub-port-specific. - SONICVPP-HLD.md: drop the added "LAG Subinterface" (PortChannel.) row. Keep the plain "Subinterface" (Ethernet0.10) row change, since the sairedis-owned LCP pair with lcp-auto-subint disabled is real and works (plain Ethernet sub-ports pass). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docs/HLD/SONICVPP-HLD.md | 1 - docs/HLD/vpp-lag.md | 34 +++------------------------------- 2 files changed, 3 insertions(+), 32 deletions(-) diff --git a/docs/HLD/SONICVPP-HLD.md b/docs/HLD/SONICVPP-HLD.md index c9f5ab7..45e67c2 100644 --- a/docs/HLD/SONICVPP-HLD.md +++ b/docs/HLD/SONICVPP-HLD.md @@ -361,7 +361,6 @@ The following table summarizes the flow from configuration to VPP/LCP API for di | **PortChannel** | sudo config portchannel add PortChannel10
    sudo config portchannel member add PortChannel10 Ethernet4 | `\|c\|SAI_OBJECT_TYPE_LAG:oid:0x2000000000095`
    `\|c\|SAI_OBJECT_TYPE_LAG_MEMBER:oid:0x1b000000000096`
    `SAI_LAG_MEMBER_ATTR_LAG_ID=oid:0x2000000000095` | `bond_create`
    `bond_add_member` | `lcp_itf_pair_add_del(BondEthernetX,beX)`

    >`itf-pair: [4] BondEthernet10 tap4099 be10 16 type tap` | `beX`
    -->
    `PortChannelX` | | **Loopback** | sudo config interface ip add Loopback0 10.0.0.1/32 | `\|c\|SAI_OBJECT_TYPE_ROUTE_ENTRY:{"dest":"10.0.0.1/32","` | `create_loopback_instance`
    `sw_interface_add_del_address` | `lcp_itf_pair_add_del(loopX,tap_LoopbackX)`

    >`itf-pair: [2] loop0 tap4098 tap_Loopback0 13 type tap` | `tap_LoopbackX`
    -->
    `LoopbackX` | | **Subinterface** | sudo config subinterface add Ethernet0.10 10 | `\|c\|SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000094`
    `SAI_ROUTER_INTERFACE_ATTR_TYPE=SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` | `create_subif`| Explicit `configure_lcp_interface(bobmX.10, EthernetX.10)` from sairedis (with `lcp-auto-subint = disabled` in startup.conf)

    >`itf-pair: [3] bobm0.10 tap4096.10 Ethernet0.10 14 type tap` | N/A, shares parent tap | -| **LAG Subinterface** | sudo config subinterface add PortChannel10.20 20 | `\|c\|SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000095`
    `SAI_ROUTER_INTERFACE_ATTR_TYPE=SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` (parent is a LAG) | `create_subif` on `BondEthernet10`, named `BondEthernet10.20` | Explicit `configure_lcp_interface(BondEthernet10.20, be10.20)` from sairedis (the SONiC L3 netdev `PortChannel10.20` is a teamd VLAN child, not a VPP tap, so cannot be the LCP host directly)

    >`itf-pair: [N] BondEthernet10.20 tap4099.20 be10.20 N type tap` | `be10.20`
    <-->
    `PortChannel10.20`
    (bidirectional `tc mirred` so ARP/L3 traffic reaches the SONiC L3 netdev that holds the IP) | diff --git a/docs/HLD/vpp-lag.md b/docs/HLD/vpp-lag.md index 3402e9e..b6e2183 100644 --- a/docs/HLD/vpp-lag.md +++ b/docs/HLD/vpp-lag.md @@ -13,8 +13,7 @@ Rev v0.2 5. [LAG and Bridging](#item-5) 6. [LACP Support](#item-6) 7. [LAG L3 Support](#item-7) -8. [LAG Subinterface](#item-8) -9. [Status](#item-9) +8. [Status](#item-8)

    @@ -26,7 +25,6 @@ Rev v0.2 |-----|------|-----------|---------| |v0.1 | 25/02/2024 | Bendrapu Balareddy (Cisco), Sameer Nanajkar (Cisco) | LAG & Bridging Support | |v0.2 | 17/12/2024 | Akeel Ali (Cisco) | LACP & L3 Support | -|v0.3 | 25/05/2026 | Bojun Feng, Lun Yue (Microsoft) | LAG Subinterface — basic sub-port path |
    @@ -37,8 +35,6 @@ Rev v0.2 This document describes the high level design of integrating LAG between SONIC and VPP. It covers: - LAG and 802.1Q bridging - LACP - - LAG L3 (PortChannel with IP) - - LAG Subinterface (PortChannel\.\)
    @@ -136,31 +132,6 @@ tc filter add dev parent ffff: \ -## LAG Subinterface - -PortChannel subinterfaces are tagged L3 interfaces on top of a LAG (`PortChannel.`), used for example by the t1-lag topology where every LAG carries multiple `/30` peering subnets, one per VLAN. - -A LAG subinterface has three coupled names that must stay consistent: `PortChannel.` (the SONiC L3 kernel netdev that teamd creates and that the IP lives on), `BondEthernet.` (the VPP sub-interface on the bond), and `be.` (the LCP host, a VLAN child of the dummy tap `be`). On `SAI_ROUTER_INTERFACE_TYPE_SUB_PORT` create, `vpp_get_hwif_name` must return `BondEthernet.`; returning the parent bond name causes every subsequent SAI call on the sub-interface (IP add, neighbor add, delete) to operate on the wrong VPP object. - -### Plumbing - -For a non-LAG sub-port (`Ethernet0.10`) the LCP host can simply be the SONiC L3 netdev itself, because it sits on top of a VPP-created tap (`tap`). That does not work for a LAG sub-port: `PortChannel.` is a teamd VLAN child, not a VPP tap, and VPP linux-cp can only punt to a kernel netdev backed by a VPP tap. - -The design pairs the VPP sub-interface with `be.` (a VLAN child of the dummy tap `be` that already exists for the parent LAG), and then bridges that LCP host to the SONiC L3 netdev `PortChannel.` with a bidirectional `tc mirred` redirect. The ingress redirect (`be.` → `PortChannel.`) carries VPP-punted ARP/ICMP frames to the netdev that holds the IP, so the kernel handles them normally. The egress redirect (`PortChannel.` → `be.`) carries kernel-originated frames into VPP via the LCP tap, where the linux-cp cross-connect forwards them to `BondEthernet.`. Both directions are mandatory: a one-direction-only bridge silently breaks LAG sub-port ping, and failure of either direction rolls back the LCP pair and VPP sub-interface so the SAI RIF create fails cleanly. - -`lcp-auto-subint` is disabled in `startup.conf.tmpl` so sairedis owns sub-port LCP pair creation end-to-end — the pair and the tc bridge go in together as part of one SAI RIF create. Leaving it enabled lets VPP create the pair on `create_subif` before the tc bridge is in place, leaving ARP replies VPP punts to `be.` with no path to `PortChannel.` (where the IP lives); the kernel ARP table on `PortChannel.` stays empty and `addNeighbor` for sub-port-over-LAG fails (the original "Bug 6: LAG sub-port 0/8 addNeighbor"). - -### RIF IP and ARP - -Sub-port connected route programming can arrive through both `ROUTER_INTERFACE` and `PORT` next-hop route events. The `ROUTER_INTERFACE` / `SUB_PORT` branch uses `vpp_add_del_intf_ip_addr()` to resolve the RIF directly from the SAI RIF OID, while the `PORT` next-hop branch keeps using `vpp_add_del_intf_ip_addr_norif()` and resolves the netdev by prefix from the host kernel. Both paths resolve LAG sub-ports to `BondEthernet.` so the IP lands on the sub-interface rather than on the parent bond. VPP returns `ADDRESS_IN_USE (-105)` when orchagent re-issues the same IP add on `config reload` or on transient connected-route remove/add cycles; the handler treats `-105` as success on add, and the transient remove path no longer clears VPP's RIF IP, so a single `config reload` does not permanently lose sub-interface IP state. - -The kernel ignores unsolicited ARP replies on a sub-port netdev by default (`arp_accept = 0`), which would otherwise break the VPP → kernel population path on the LCP host. `vslib/vpp` sets `net.ipv4.conf..arp_accept = 1` on RIF create, and pre-warms the VPP neighbor table on RIF IP add so the first packet does not have to wait for resolution. - -
    -
    - - - ## Status @@ -172,7 +143,7 @@ Ping between PortChannels | Coded solution using `tc` utility to redirect traffi Testing | Bring-up, ping of PortChannel with 2 members, add/remove members, v4/v6/multiple-members, multiple-portchannels, L2 traffic | Sonic-mgmt t1-28-lag testing | Brought-up t1-28-lag TOPO. Ran LAG tests |
  • Debug and fix failing LAG TCs
    • Hashing algo selection | Coded using XOR & L3L4 | (Optional, TBD) Ability to switchover between L2L3 and L3L4 depending on presence of IP (or other scheme) -(T0-lag) PortChannel Subinterfaces |
  • Hwif naming `BondEthernet.`
  • Explicit LCP pair `BondEthernet.` ↔ `be.` plus bidirectional `tc mirred` bridge `be.` ↔ `PortChannel.` (with `lcp-auto-subint=disabled` so sairedis is the sole producer of the pair)
  • RIF IP programming on VPP sub-interface, idempotent on `-105`, preserved across transient remove cycles
  • `arp_accept=1` and VPP neighbor pre-warm on sub-port RIF create
  • Representative `t1-lag-vpp` sub-port smoke tests PASS
    • |
  • SVI variants (sonic-buildimage #26936) and IPinIP tunneling (sairedis #1860) — separate features
    • +(Phase 2 - T0-lag) PortChannel Subinterfaces | Identified changes required to provision PortChannel subintf in VPP and apply IP |
  • Ping fails
  • May require redesign to add PortChannel support to existing interface APIs
  • Bring-up t0-lag topo and run LAG tests
    • @@ -180,3 +151,4 @@ Hashing algo selection | Coded using XOR & L3L4 | (Optional, TBD) Ability to swi [SONiC system architecture](https://github.com/sonic-net/SONiC/wiki/Architecture)\ [What is VPP](https://s3-docs.fd.io/vpp/23.06/) + From 2be196a29625c3b021efb443b628f3845efe67b9 Mon Sep 17 00:00:00 2001 From: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> Date: Mon, 15 Jun 2026 03:32:47 +0000 Subject: [PATCH 10/10] [linux-cp]: fix stale lcp-auto-subint comment (drop removed tc bridge) The comment above the disabled lcp-auto-subint directive still described the removed LAG sub-port tc-mirred bridge and framed the disable as LAG-specific. With the tc bridge gone (per review on this PR) and LAG sub-port punt deferred to a future VPP node, restate the actual reason: lcp-auto-subint is disabled so sairedis owns sub-port LCP pair creation explicitly (create_subif + configure_lcp_interface), which would otherwise collide with VPP's auto-pair. Directive itself (lcp-auto-subint disabled + lcp-sync) is unchanged; it is required so plain Ethernet sub-ports get their LCP pair from sairedis. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Lun Yue <17232861+lunyue-ms@users.noreply.github.com> --- docker-sonic-vpp/conf/startup.conf.tmpl | 10 ++++------ docker-syncd-vpp/conf/startup.conf.tmpl | 10 ++++------ 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/docker-sonic-vpp/conf/startup.conf.tmpl b/docker-sonic-vpp/conf/startup.conf.tmpl index 6948cc5..0e35460 100644 --- a/docker-sonic-vpp/conf/startup.conf.tmpl +++ b/docker-sonic-vpp/conf/startup.conf.tmpl @@ -301,12 +301,10 @@ plugins { # Why not support sub interfaces linux-cp { - # lcp-auto-subint disabled: auto-pair "BondEthernet." <-> "be." - # punts ARP to be., but SONiC writes the sub-port IP on - # "PortChannel." (separate teamd vlan netdev), so the kernel drops - # the unsolicited ARP and the VPP neighbor table never gets populated. - # Disabled so sairedis (vpp_create_router_interface, SUB_PORT path) creates the - # LCP pair + tc mirred bridge between be. and PortChannel.. + # lcp-auto-subint disabled so sairedis owns sub-port LCP pair creation + # explicitly (vpp_create_router_interface, SUB_PORT path: create_subif + + # configure_lcp_interface). With it enabled VPP would also auto-create the + # pair, colliding with the explicit one. # lcp-auto-subint lcp-sync } diff --git a/docker-syncd-vpp/conf/startup.conf.tmpl b/docker-syncd-vpp/conf/startup.conf.tmpl index a6991d9..cf39b61 100644 --- a/docker-syncd-vpp/conf/startup.conf.tmpl +++ b/docker-syncd-vpp/conf/startup.conf.tmpl @@ -289,12 +289,10 @@ plugins { # Why not support sub interfaces linux-cp { - # lcp-auto-subint disabled: auto-pair "BondEthernet." <-> "be." - # punts ARP to be., but SONiC writes the sub-port IP on - # "PortChannel." (separate teamd vlan netdev), so the kernel drops - # the unsolicited ARP and the VPP neighbor table never gets populated. - # Disabled so sairedis (vpp_create_router_interface, SUB_PORT path) creates the - # LCP pair + tc mirred bridge between be. and PortChannel.. + # lcp-auto-subint disabled so sairedis owns sub-port LCP pair creation + # explicitly (vpp_create_router_interface, SUB_PORT path: create_subif + + # configure_lcp_interface). With it enabled VPP would also auto-create the + # pair, colliding with the explicit one. # lcp-auto-subint lcp-sync }