Add HFT harmonizer configuration

Signed-off-by: Ze Gan <ganze718@gmail.com>

Author: Pterosaur

doc/high-frequency-telemetry/high-frequency-telemetry-hld.md

@@ -26,7 +26,8 @@
   - [7.4. Config DB](#74-config-db)
     - [7.4.1. DEVICE\_METADATA](#741-device_metadata)
     - [7.4.2. HIGH\_FREQUENCY\_TELEMETRY\_PROFILE](#742-high_frequency_telemetry_profile)
-    - [7.4.3. HIGH\_FREQUENCY\_TELEMETRY\_GROUP](#743-high_frequency_telemetry_group)
+    - [7.4.3. HIGH\_FREQUENCY\_TELEMETRY\_HARMONIZER](#743-high_frequency_telemetry_harmonizer)
+    - [7.4.4. HIGH\_FREQUENCY\_TELEMETRY\_GROUP](#744-high_frequency_telemetry_group)
   - [7.5. StateDb](#75-statedb)
     - [7.5.1. HIGH\_FREQUENCY\_TELEMETRY\_SESSION](#751-high_frequency_telemetry_session)
   - [7.6. Work Flow](#76-work-flow)
@@ -52,6 +53,7 @@
 | --- | ---------- | ------ | ------------------ |
 | 0.1 | 09/06/2024 | Ze Gan | Initial version    |
 | 0.2 | 03/01/2025 | Janet Cui | Initial version    |
+| 0.3 | 06/21/2026 | Ze Gan | Add harmonizer configuration |
 
 ## 2. Scope
 
@@ -130,6 +132,7 @@ flowchart LR
     asic[\ASIC\]
 
     config_db --HIGH_FREQUENCY_TELEMETRY_PROFILE
+                HIGH_FREQUENCY_TELEMETRY_HARMONIZER
                 HIGH_FREQUENCY_TELEMETRY_GROUP--> hft_orch
     state_db --HIGH_FREQUENCY_TELEMETRY_SESSION--> counter_syncd
     hft_orch --HIGH_FREQUENCY_TELEMETRY_SESSION--> state_db
@@ -169,7 +172,7 @@ flowchart LR
     swss_act((Swss actor: Handle swss message))
     netlink_act((Netlink actor: Receive netlink message from kernel))
     ipfix_act((Ipfix actor: Handle IPFix message))
-    harmonize_act((Harmonize actor: Unifies sampling and handles data rollover))
+    harmonize_act((Harmonize actor: Aggregates samples and handles data rollover))
     cdb_act((Counter DB actor: Store counters to counter DB))
     otel_act((OpenTelemetry actor: Send counters to OpenTelemetry collector))
     cdb[(Counter DB)]
@@ -186,13 +189,22 @@ flowchart LR
 
 ```
 
+The Harmonize actor applies optional per-profile harmonization after IPFIX records are parsed and before counters are exported to Counter DB or OpenTelemetry. A profile selects a harmonizer by setting `HIGH_FREQUENCY_TELEMETRY_PROFILE.harmonizer`; if no harmonizer is selected, Counter Syncd forwards the lower-layer reported samples without extra aggregation, rollover correction, or heatmap handling.
+
+A harmonizer supports the following methods and can be extended with more methods later:
+
+- Reporting rate aggregation: aggregates the lower-layer reported samples into the configured harmonizer reporting interval. The unit is microseconds. If `reporting_rate` is not configured, the harmonizer uses the lower-layer reporting interval and does not aggregate samples.
+- Data rollover: enables rollover correction for the configured group and counter pairs. The list is empty by default, so no counters are corrected for rollover unless configured.
+- Heatmap data: marks the configured group and counter pairs as heatmap data. The list is empty by default.
+
 #### 7.1.2. High frequency telemetry Orch
 
 The `High frequency telemetry Orch` is a new object within the Orchagent. It has following primary duties:
 
 1. Maintain the TAM SAI objects according to the high frequency telemetry configuration in the config DB.
 2. Generate a unique template ID for each high frequency telemetry profile to ensure distinct identification and management.
 3. Register and activate streams on counter syncd.
+4. Resolve the harmonizer configuration referenced by each profile and publish it with the session metadata for Counter Syncd.
 
 `High frequency telemetry Orch` leverages `tam_counter_subscription` objects to bind monitoring objects, such as ports, buffers, or queues, to streams. Therefore, this orch must ensure that the lifecycle of `tam_counter_subscription` objects is within the lifecycle of their respective monitoring objects.
 
@@ -509,6 +521,7 @@ HIGH_FREQUENCY_TELEMETRY_PROFILE|{{profile_name}}
     "poll_interval": {{uint32}}
     "otel_endpoint": {{string of endpoint}} (Optional)
     "otel_certs": {{string of path}} (Optional)
+    "harmonizer": {{string of harmonizer name}} (Optional)
 ```
 
 ```
@@ -520,9 +533,37 @@ otel_endpoint      = string ; The endpoint of OpenTelemetry collector. E.G. 192.
                      It will use the local OpenTelemetry collector if this value isn't provided.
 otel_certs         = string ; The path of certificates for OpenTelemetry collector. E.G. /etc/sonic/otel/cert.private
                      If this value isn't provided, we will use a non-secure channel.
+harmonizer         = string ; The optional name of the HIGH_FREQUENCY_TELEMETRY_HARMONIZER entry that this profile applies.
+                     If this value isn't provided, no harmonizer configuration is applied.
+```
+
+#### 7.4.3. HIGH_FREQUENCY_TELEMETRY_HARMONIZER
+
+```
+HIGH_FREQUENCY_TELEMETRY_HARMONIZER|{{harmonizer_name}}
+    "reporting_rate": {{uint32}} (Optional)
+    "data_rollover_flags": {{list of group and counter pair}} (Optional)
+    "heatmap_counters": {{list of group and counter pair}} (Optional)
+```
+
+```
+key                  = HIGH_FREQUENCY_TELEMETRY_HARMONIZER|harmonizer_name a string as the identifier of harmonizer configuration
+; field              = value
+reporting_rate       = uint32 ; The reporting interval after harmonization, unit microseconds.
+                       It aggregates lower-layer reported samples within each reporting window.
+                       If this value isn't provided, the harmonizer uses the lower-layer reporting interval and does not aggregate samples.
+data_rollover_flags  = A list of group and counter pairs that require rollover correction.
+                       The syntax of each entry is group_name|object_counter.
+                       In CONFIG_DB, multiple entries are encoded as group_name1|object_counter1,group_name2|object_counter2.
+                       The group_name must match HIGH_FREQUENCY_TELEMETRY_GROUP.group_name, and object_counter must be valid for that group.
+                       Examples are PORT|IF_IN_UCAST_PKTS and QUEUE|DROPPED_PACKETS.
+                       The default value is empty, meaning no counters are corrected for rollover.
+heatmap_counters     = A list of group and counter pairs that should be treated as heatmap data.
+                       The syntax is the same as data_rollover_flags, for example PORT|IF_IN_UCAST_PKTS and QUEUE|DROPPED_PACKETS.
+                       The default value is empty.
 ```
 
-#### 7.4.3. HIGH_FREQUENCY_TELEMETRY_GROUP
+#### 7.4.4. HIGH_FREQUENCY_TELEMETRY_GROUP
 
 ```
 HIGH_FREQUENCY_TELEMETRY_GROUP|{{profile_name}}|{{group_name}}
@@ -543,7 +584,7 @@ object_names    = A list of object name.
 object_counters = A list of stats of object;
 ```
 
-For the schema of `HIGH_FREQUENCY_TELEMETRY_GROUP`, please refer to its [YANG model](sonic-high-frequency-telemetry.yang).
+For the schemas of high frequency telemetry config tables, please refer to their [YANG model](sonic-high-frequency-telemetry.yang).
 
 ### 7.5. StateDb
 
@@ -556,6 +597,7 @@ HIGH_FREQUENCY_TELEMETRY_SESSION|{{profile_name}}|{{group_name}}
     "object_ids": {{list of uint16_t}}
     "session_type": {{ipfix}}
     "session_config": {{binary array}}
+    "harmonizer_config": {{serialized harmonizer configuration}} (Optional)
     "config_version": {{uint32_t}}
 ```
 
@@ -571,6 +613,9 @@ object_ids          = A list of object ID;
 session_type        = ipfix ; Specified the session type.
 session_config      = binary array;
                       If the session type is IPFIX, This field stores the IPFIX template to interpret the message of this session.
+harmonizer_config   = string;
+                      The serialized HIGH_FREQUENCY_TELEMETRY_HARMONIZER configuration applied by this session.
+                      If this value isn't provided, Counter Syncd does not apply harmonization to this session.
 config_version      = uint32_t; Indicates which version is being used. The default value is 0.
                       This value will be increased once the new config was applied by CounterSyncd.
 ```
@@ -606,6 +651,7 @@ sequenceDiagram
     hft_orch ->> syncd: Initialize <br/>HOSTIF<br/>TAM_TRANSPORT<br/>TAM_collector<br/>
 
     config_db ->> hft_orch: HIGH_FREQUENCY_TELEMETRY_PROFILE
+    config_db ->> hft_orch: HIGH_FREQUENCY_TELEMETRY_HARMONIZER
     config_db ->> hft_orch: HIGH_FREQUENCY_TELEMETRY_GROUP
     port_orch ->> hft_orch: Port/Queue/Buffer ... object
 
@@ -643,6 +689,7 @@ sequenceDiagram
     end
     loop Receive IPFIX message of stats from genetlink
         alt Have this template of IPFIX been registered?
+            counter ->> counter: Apply harmonizer configuration
             counter ->> otel: Push message to OpenTelemetry Collector
         else
             counter ->> counter: Discard this message
@@ -673,6 +720,12 @@ sudo config high_frequency_telemetry profile add $profile_name --stream_state=$s
 # Change stream state
 sudo config high_frequency_telemetry profile set $profile_name --stream_state=$stream_state
 
+# Bind a harmonizer to a profile
+sudo config high_frequency_telemetry profile set $profile_name --harmonizer=$harmonizer_name
+
+# Add a harmonizer
+sudo config high_frequency_telemetry harmonizer add $harmonizer_name --reporting_rate=$reporting_rate --data_rollover_flags="$group_name1|$object_counter1,$group_name2|$object_counter2" --heatmap_counters="$group_name3|$object_counter3"
+
 # Add a monitor group
 sudo config high_frequency_telemetry group "$profile|$group_name" --object_names="$object1,$object2" --object_counters="$object_counters1,$object_counters2"
 

doc/high-frequency-telemetry/sonic-high-frequency-telemetry.yang

@@ -41,16 +41,54 @@ module sonic-high-frequency-telemetry {
                     }
                 }
 
-                leaf poll_interval {
-                    mandatory true;
-                    description "The interval to poll counter, unit milliseconds.";
-                    type uint32;
-                }
-
-            }
-        }
-
-        container HIGH_FREQUENCY_TELEMETRY_GROUP {
+                leaf poll_interval {
+                    mandatory true;
+                    description "The interval to poll counter, unit microseconds.";
+                    type uint32;
+                }
+
+                leaf harmonizer {
+                    description "Optional harmonizer configuration applied by this profile.";
+                    type leafref {
+                        path "/sonic-high-frequency-telemetry:sonic-high-frequency-telemetry/HIGH_FREQUENCY_TELEMETRY_HARMONIZER/HIGH_FREQUENCY_TELEMETRY_HARMONIZER_LIST/name";
+                    }
+                }
+
+            }
+        }
+
+        container HIGH_FREQUENCY_TELEMETRY_HARMONIZER {
+            description "HIGH_FREQUENCY_TELEMETRY_HARMONIZER part of config_db.json";
+            list HIGH_FREQUENCY_TELEMETRY_HARMONIZER_LIST {
+                key "name";
+
+                leaf name {
+                    type string {
+                        length 1..128;
+                    }
+                }
+
+                leaf reporting_rate {
+                    description "The reporting interval after harmonization, unit microseconds.";
+                    type uint32;
+                }
+
+                leaf-list data_rollover_flags {
+                    description "List of group and counter pairs that require rollover correction.";
+                    type string;
+                    must "re-match(current(), '(PORT|BUFFER_POOL|INGRESS_PRIORITY_GROUP|QUEUE)\\|[^|]+')";
+                }
+
+                leaf-list heatmap_counters {
+                    description "List of group and counter pairs that should be treated as heatmap data.";
+                    type string;
+                    must "re-match(current(), '(PORT|BUFFER_POOL|INGRESS_PRIORITY_GROUP|QUEUE)\\|[^|]+')";
+                }
+
+            }
+        }
+
+        container HIGH_FREQUENCY_TELEMETRY_GROUP {
             description "HIGH_FREQUENCY_TELEMETRY_GROUP part of config_db.json";
             list HIGH_FREQUENCY_TELEMETRY_GROUP_LIST {
                 key "profile_name group_name";