[featured]: Release delayed features on multi-ASIC via per-ASIC PortInitDone

[william8545]

Problem

On multi-ASIC, PortInitDone is published per-ASIC in each namespace's APPL_DB, but featured only watched the host APPL_DB. It therefore never observed the marker and released the delayed features (lldp, gnmi, mgmt-framework, snmp, sflow, pmon) only after the 180s fallback timeout, slowing every reload/reboot.

Fix

• Subscribe each port-bearing namespace's PORT_TABLE and release delayed features once all of them report PortInitDone (quorum).
• Detect port-bearing namespaces by PORT table presence and reconcile the quorum against the connectors actually established, so a failed connector degrades to the timeout backstop instead of blocking release forever.
• Key subscriptions and callbacks by (namespace, table) to support per-namespace listeners.
• Single-ASIC behavior is unchanged.

Tests

Unit tests for the quorum, re-release after swss restart, out-of-quorum namespaces, non-SET/other keys, the reconcile path, and the multi/single-ASIC register_callbacks wiring.

Verification

Verified on a multi-ASIC setup: delayed features release via the port-init path shortly after featured starts (instead of the 180s timeout), with all expected containers up and no regression on restart or cold reboot.

Summary by CodeRabbit

Release Notes

• New Features

  • Multi-ASIC systems now coordinate service initialization through quorum tracking to ensure all required ASICs signal readiness before delayed services are enabled.

• Tests

  • Added comprehensive test coverage validating quorum-based service initialization behavior, including scenarios for service re-enablement after restarts and handling of out-of-quorum namespaces in multi-ASIC deployments.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: nvidia-sonic/coderabbit/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 28d6f763-e27e-440e-a50b-59ff972a2b72

📥 Commits

Reviewing files that changed from the base of the PR and between c5bbbe8 and e56a73a.

📒 Files selected for processing (2)

• scripts/featured
• tests/featured/featured_test.py

---

📝 Walkthrough

Walkthrough

This PR implements multi-ASIC PortInitDone quorum tracking for delayed service enablement. Previously, delayed services were enabled immediately upon any single PortInitDone event. Now, they are enabled only after all participating namespaces have reported, via per-namespace subscription with namespace-aware event dispatch.

Changes

Multi-ASIC Delayed Service Enablement

Layer / File(s)	Summary
Quorum state initialization scripts/featured	Adds port_init_quorum_ns and pending_port_init_ns tracking sets in FeatureHandler. During startup, probes each per-namespace CONFIG_DB for PORT table presence to determine which namespaces participate in the quorum.
Port listener and quorum-aware service enabling scripts/featured	port_listener now accepts namespace parameter. On single-ASIC, any PortInitDone triggers delayed enabling immediately. On multi-ASIC, removes the reporting namespace from pending set and enables delayed services only when all subscribed namespaces have reported; later events re-trigger idempotently.
Multi-ASIC subscription and event dispatch scripts/featured	subscribe and register_callbacks extended to carry namespace parameter. Multi-ASIC creates per-namespace APPL_DB connectors for quorum namespaces and subscribes to PORT in each namespace with namespace-bound callbacks. Event dispatch loop updated to use (namespace, table) keys for callback lookup and routing.
Quorum and subscription behavior tests tests/featured/featured_test.py	Adds test helpers and new cases validating multi-ASIC quorum completion (all namespaces required), re-enabling after swss restart, ignoring out-of-quorum namespaces and non-SET events, and per-namespace vs. single-namespace subscription behavior in register_callbacks.

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately describes the main change: implementing multi-ASIC PortInitDone-based release of delayed services instead of the previous single-ASIC or timeout-based approach.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Dco Signoff	✅ Passed	The commit e56a73a contains a valid DCO signoff matching the author: "Signed-off-by: William Tsai willtsai@nvidia.com"

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sw-r2d2-bot]

❌ SONiC CI #4533 finished with status: FAILURE