Skip to content

Add Timestamp Display for FEC Histogram Counters#4513

Open
bobby-nexthop wants to merge 2 commits into
sonic-net:masterfrom
nexthop-ai:bobby.fec-histogram-timestamps-clean
Open

Add Timestamp Display for FEC Histogram Counters#4513
bobby-nexthop wants to merge 2 commits into
sonic-net:masterfrom
nexthop-ai:bobby.fec-histogram-timestamps-clean

Conversation

@bobby-nexthop

Copy link
Copy Markdown
Contributor

What I did

FEC histogram counters (S0–S15) are polled periodically by FlexCounter, but users have no way to know when a given bin value last changed. Adding per-bin timestamps lets operator determine the freshness of each FEC codeword error bucket.

Display the time each FEC histogram counter was last updated by reading the timestamp from the COUNTERS DB and formatting it as a human-readable datetime. Includes --relative-timestamp flag and corresponding test updates.

How I did it

How to verify it

  • Configure FlexCounter to poll FEC histogram stats on a port.
  • Verify that COUNTERS_DB entries for the port include
  • Confirm timestamps are only updated when the corresponding counter value changes (not on every poll cycle).

Previous command output (if the output of a command-line utility has changed)

admin@device:~$ show int counters fec-histogram Ethernet504
Symbol Errors Per Codeword       Codewords 
----------------------------  ------------ 
BIN0                          314158685800
BIN1                           22519111278 
BIN2                            1240501695  
BIN3                              54230751 
BIN4                               2117541 
BIN5                                 76776 
BIN6                                  2789 
BIN7                                    89 
BIN8                                    10 
BIN9                                     0  
BIN10                                    0 
BIN11                                    0 
BIN12                                    0  
BIN13                                    0  
BIN14                                    0  
BIN15                                    0  

New command output (if the output of a command-line utility has changed)

admin@device:~$ show int counters fec-histogram Ethernet504
Symbol Errors Per Codeword       Codewords  Last Updated
----------------------------  ------------  -----------------------
BIN0                          314158685800  2026-03-26 01:39:56
BIN1                           22519111278  2026-03-26 01:39:56
BIN2                            1240501695  2026-03-26 01:39:56
BIN3                              54230751  2026-03-26 01:39:56
BIN4                               2117541  2026-03-26 01:39:56
BIN5                                 76776  2026-03-26 01:39:56
BIN6                                  2789  2026-03-26 01:39:56
BIN7                                    89  2026-03-26 01:38:56
BIN8                                    10  2026-03-26 01:37:56
BIN9                                     0  2026-03-26 01:21:39
BIN10                                    0  2026-03-26 01:21:39
BIN11                                    0  2026-03-26 01:21:39
BIN12                                    0  2026-03-26 01:21:39
BIN13                                    0  2026-03-26 01:21:39
BIN14                                    0  2026-03-26 01:21:39
BIN15                                    0  2026-03-26 01:21:39
admin@gold309:~$ show int counters fec-histogram Ethernet0 --relative-timestamp
Symbol Errors Per Codeword         Codewords  Last Updated         Relative Time
----------------------------  --------------  -------------------  ---------------
BIN0                          65499246582534  2026-05-04 13:24:32  0 seconds ago
BIN1                               123940540  2026-05-04 13:24:32  0 seconds ago
BIN2                                 1421967  2026-05-04 13:21:53  2 minutes ago
BIN3                                   63761  2026-04-28 23:15:38  5 days ago
BIN4                                    3165  2026-04-28 23:15:37  5 days ago
BIN5                                     202  2026-04-28 23:15:37  5 days ago
BIN6                                      37  2026-04-28 22:00:39  5 days ago
BIN7                                      18  2026-04-28 17:46:40  5 days ago
BIN8                                       8  2026-04-28 17:46:40  5 days ago
BIN9                                       8  2026-04-28 17:46:40  5 days ago
BIN10                                      9  2026-04-28 17:46:40  5 days ago
BIN11                                      6  2026-04-28 17:46:40  5 days ago
BIN12                                      6  2026-04-28 17:46:40  5 days ago
BIN13                                      6  2026-04-28 17:46:40  5 days ago
BIN14                                      6  2026-04-28 17:46:40  5 days ago
BIN15                                      4  2026-04-28 17:46:40  5 days ago

Display the time each FEC histogram counter was last updated by reading
the timestamp from the COUNTERS DB and formatting it as a human-readable
datetime. Includes --relative-timestamp flag and corresponding test updates.

Signed-off-by: Bobby McGonigle <bobby@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@prgeor prgeor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bobby-nexthop This is so useful enhancement. For completeness do you mind to update the doc

Image

@prgeor

prgeor commented May 11, 2026

Copy link
Copy Markdown
Contributor

@bobby-nexthop what about the case for gearbox? @arpit-nexthop @bobby-nexthop

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the show interfaces counters fec-histogram CLI in sonic-utilities by displaying per-bin “Last Updated” timestamps (sourced from COUNTERS_DB) and adding an optional --relative-timestamp view to help operators assess histogram freshness.

Changes:

  • Extend FEC histogram output to include a per-bin “Last Updated” timestamp (from *_TIMESTAMP COUNTERS_DB fields).
  • Add --relative-timestamp option to display human-readable relative age (“X minutes/days ago”).
  • Update unit tests and mock COUNTERS_DB fixtures to include timestamp fields and validate the new output.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
show/interfaces/__init__.py Adds timestamp/relative-time columns and a new CLI flag for FEC histogram display.
utilities_common/general.py Introduces a shared helper to format relative time from epoch-ms timestamps.
tests/portstat_test.py Updates expected fec-histogram output and adds a new test for --relative-timestamp.
tests/portstat_db/counters_db.json Adds an example *_TIMESTAMP counter key for unit test data.
tests/mock_tables/counters_db.json Mirrors the timestamp fixture change for mock table data.

Comment thread show/interfaces/__init__.py
Comment thread show/interfaces/__init__.py Outdated
Comment on lines +893 to +897
if timestamp_ms:
timestamp_sec = int(timestamp_ms) / 1000.0
timestamp_str = datetime.fromtimestamp(timestamp_sec).strftime('%Y-%m-%d %H:%M:%S')
else:
timestamp_str = 'N/A'

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repo in other places always uses local time and not UTC, so will keep this as localtime. However the 0 validation is fair, adding that guard

Comment thread show/interfaces/__init__.py
Comment on lines +80 to +83
str: Relative time string (e.g. "5 minutes ago", "2 days ago"), or "N/A" if in the future
'''
now = datetime.now()
ts = datetime.fromtimestamp(int(timestamp_ms) / 1000.0)
Comment thread tests/portstat_test.py Outdated
Comment on lines +520 to +537
def test_show_intf_counters_fec_histogram_relative_timestamp(self):
runner = CliRunner()
result = runner.invoke(
show.cli.commands["interfaces"].commands["counters"].commands["fec-histogram"],
["--relative-timestamp", "Ethernet0"])
print(result.exit_code)
print(result.output)
assert result.exit_code == 0
# Verify the extra column header is present
assert 'Relative Time' in result.output
# BIN0 has a timestamp so it should show a relative time (e.g. "days ago")
assert 'ago' in result.output
# BIN1 has no timestamp so it should show N/A in the relative column
lines = result.output.strip().split('\n')
# Header line, separator line, then 16 data lines
bin1_line = lines[3] # BIN1 is the second data row
# BIN1 should have two N/A entries (Last Updated and Relative Time)
assert bin1_line.count('N/A') == 2
@prgeor

prgeor commented May 12, 2026

Copy link
Copy Markdown
Contributor

@bobby-nexthop please address copilot comment too

Signed-off-by: Bobby McGonigle <bobby@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

error_value = asic_db_kvp.get(counter_key, '0')
timestamp_ms = asic_db_kvp.get(timestamp_key, None)

if timestamp_ms and int(timestamp_ms) > 0:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bobby-nexthop @arpit-nexthop who is updating the timestamp when the BIN value changes? orchagent?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants