Skip to content

[doc]: Enhance HFT document with aggregator and stats#2079

Open
Pterosaur wants to merge 5 commits into
sonic-net:masterfrom
Pterosaur:enhance_hft
Open

[doc]: Enhance HFT document with aggregator and stats#2079
Pterosaur wants to merge 5 commits into
sonic-net:masterfrom
Pterosaur:enhance_hft

Conversation

@Pterosaur

@Pterosaur Pterosaur commented Oct 5, 2025

Copy link
Copy Markdown
Contributor

What I did

Enhanced the High Frequency Telemetry HLD with aggregator support and related schema updates.

This PR documents:

  • The CounterSyncd aggregator actor and its position between IPFIX parsing and Counter DB / OTEL reporting.
  • HIGH_FREQUENCY_TELEMETRY_AGGREGATOR as a new CONFIG_DB table for aggregator methods.
  • Optional HIGH_FREQUENCY_TELEMETRY_PROFILE.aggregator binding so a profile can select an aggregator configuration.
  • Aggregator reporting_rate aggregation, with microseconds as the unit and no aggregation when unset.
  • rollover_counters and heatmap_counters as comma-separated GROUP|COUNTER lists, using the same list format as object_names / object_counters.
  • YANG model updates, including group-specific counter validation for aggregator counter lists.

Why I did it

HFT needs aggregation configuration to be independent from the base telemetry profile and extensible for future methods. Keeping aggregator configuration in a dedicated table lets profiles opt in by reference and leaves room for later aggregation methods without changing the profile schema each time.

Feature PR tracking

Repository PR Scope Status
sonic-net/sonic-buildimage #28009 HFT aggregator YANG schema and YANG model tests PR status
sonic-net/sonic-utilities #4631 HFT aggregator config/show CLI support PR status
sonic-net/sonic-swss #4691 countersyncd aggregator actor and reporting_rate aggregation PR status

How I verified it

Validated the YANG syntax locally with pyang using minimal stubs for imported SONiC modules:

python -m pyang -p "C:\Users\zegan\AppData\Local\Temp\opencode\hft-yang-stubs" doc/high-frequency-telemetry/sonic-high-frequency-telemetry.yang

Also ran local instance-level checks for rollover_counters / heatmap_counters validation:

  • Valid multi-entry list with PORT, BUFFER_POOL, INGRESS_PRIORITY_GROUP, and QUEUE counters passed.
  • Invalid group/counter combinations failed as expected, including PORT|DROPPED_PACKETS, QUEUE|IF_IN_UCAST_PKTS, ACL|PACKETS, and missing | separator.

Details if related

The document describes future aggregator methods, but implementation can land in phases. The current linked sonic-swss implementation PR covers the first phase: reporting_rate aggregation.

Signed-off-by: Ze Gan <ganze718@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

lihuay
lihuay previously approved these changes Nov 20, 2025
@Pterosaur Pterosaur changed the title [doc]: Enhance HFT document [doc]: Enhance HFT document with harmonizer and stats Jun 21, 2026
Signed-off-by: Ze Gan <ganze718@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

Signed-off-by: Ze Gan <ganze718@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

Signed-off-by: Ze Gan <ganze718@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
No pipelines are associated with this pull request.

- [7.4.1. DEVICE\_METADATA](#741-device_metadata)
- [7.4.2. HIGH\_FREQUENCY\_TELEMETRY\_PROFILE](#742-high_frequency_telemetry_profile)
- [7.4.3. HIGH\_FREQUENCY\_TELEMETRY\_GROUP](#743-high_frequency_telemetry_group)
- [7.4.3. HIGH\_FREQUENCY\_TELEMETRY\_HARMONIZER](#743-high_frequency_telemetry_harmonizer)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

harmonizer is too tricky, I think we should make the name more revealing.

i would recommend: post-processor / aggregator. something like this.

The group_name must match HIGH_FREQUENCY_TELEMETRY_GROUP.group_name, and object_counter must be valid for that group.
An example is PORT|IF_IN_UCAST_PKTS,QUEUE|DROPPED_PACKETS.
The default value is empty, meaning no counters are corrected for rollover.
heatmap_counters = A comma-separated list of group and counter pairs that should be treated as heatmap data.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heatmap needs to support bucket config. needs to add this one in.

Signed-off-by: Ze Gan <ganze718@gmail.com>
@Pterosaur Pterosaur changed the title [doc]: Enhance HFT document with harmonizer and stats [doc]: Enhance HFT document with aggregator and stats Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants