Skip to content

Revert "Upgrade PMON to trixie"#27668

Closed
xwjiang-ms wants to merge 1 commit into
masterfrom
revert-26923-pmon_trixie_migration
Closed

Revert "Upgrade PMON to trixie"#27668
xwjiang-ms wants to merge 1 commit into
masterfrom
revert-26923-pmon_trixie_migration

Conversation

@xwjiang-ms

@xwjiang-ms xwjiang-ms commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Reverts #26923

#26923 (PMON → Trixie) breaks dualtor pretest from 2026-05-29 onward.
show mux status returns rc=1 with could not retrieve key state value for port EthernetX inside table MUX_CABLE_TABLE — the row exists in STATE_DB, but the state field is never written.

Root cause: MUX_CABLE_TABLE:state is published by y_cable_helper (xcvrd subprocess) inside the pmon container. Moving pmon from Bookworm to Trixie bumps Python 3.11 → 3.13 (where imp is removed — same breakage already patched in sonic-bmcctld via #27557), flips librrd8librrd8t64, jumps grpcio 1.51 → 1.67, and changes the
smartmontools source. Anything in xcvrd / y_cable_helper that wasn't rebuilt against Trixie's libs ImportErrors at container start, so mux state never propagates.

Failing baseline:
https://elastictest.org/scheduler/testplan/6a19eb5c9262108863d7b5bd?testcase=test_pretest.py%7C%7C%7Cvms-kvm-dual-t0_australiaeast_213252&type=console

Verify by using the image built by this PR, testplan link:
https://elastictest.org/scheduler/testplan/6a1ec649729d944bd21c8fb5?testcase=test_pretest.py%7C%7C%7Cvms-kvm-dual-t0_australiaeast_216606&type=console

PMON Trixie migration to be re-attempted once xcvrd / y_cable_helper
Python 3.13 compatibility is fixed.

Copilot AI review requested due to automatic review settings June 2, 2026 06:50
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reverts the previous change that moved the PMON (platform-monitor) container build to Debian trixie, restoring it to the Debian bookworm build flow used by the rest of the bookworm image pipeline.

Changes:

  • Switch PMON build dependencies/load-dockers from docker-config-engine-trixie back to docker-config-engine-bookworm.
  • Update PMON Dockerfile base image and Debian package names to the bookworm equivalents (e.g., librrd8 vs librrd8t64) and revert grpcio/grpcio-tools pins to the bookworm-aligned versions.
  • Move PMON registration from SONIC_TRIXIE_DOCKERS to SONIC_BOOKWORM_DOCKERS.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
rules/docker-platform-monitor.mk Reverts PMON’s build to bookworm by restoring bookworm config-engine deps and docker registration variables.
dockers/docker-platform-monitor/Dockerfile.j2 Reverts PMON’s base image/packages to bookworm and adjusts package/version pins accordingly.

Comment thread dockers/docker-platform-monitor/Dockerfile.j2
@lolyu
lolyu requested a review from saiarcot895 June 3, 2026 02:13

@saiarcot895 saiarcot895 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the actual error here when running show mux status? Were there any processes not running?

@xwjiang-ms xwjiang-ms closed this Jun 3, 2026
@xwjiang-ms xwjiang-ms reopened this Jun 3, 2026
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@lolyu

lolyu commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Hi @saiarcot895, the current internal dualtor image is broken, here is the error:

2026 Jun  2 04:07:48.448775 str3-8101c1-07 INFO pmon#supervisord 2026-06-02 04:07:48,448 WARN exited: ycabled (exit status 1; not expected)
2026 Jun  2 04:07:51.512461 str3-8101c1-07 INFO pmon#supervisord 2026-06-02 04:07:51,511 INFO spawned: 'ycabled' with pid 120
2026 Jun  2 04:07:51.519360 str3-8101c1-07 INFO pmon#supervisord: ycabled nice: cannot set niceness: Permission denied
2026 Jun  2 04:07:51.740659 str3-8101c1-07 INFO pmon#supervisord: ycabled Traceback (most recent call last):
2026 Jun  2 04:07:51.741317 str3-8101c1-07 INFO pmon#supervisord: ycabled   File "/usr/local/bin/ycabled", line 5, in <module>
2026 Jun  2 04:07:51.741345 str3-8101c1-07 INFO pmon#supervisord: ycabled     from ycable.ycable import main
2026 Jun  2 04:07:51.741366 str3-8101c1-07 INFO pmon#supervisord: ycabled   File "/usr/local/lib/python3.13/dist-packages/ycable/ycable.py", line 21, in <module>
2026 Jun  2 04:07:51.741381 str3-8101c1-07 INFO pmon#supervisord: ycabled     from .ycable_utilities import y_cable_helper
2026 Jun  2 04:07:51.741398 str3-8101c1-07 INFO pmon#supervisord: ycabled   File "/usr/local/lib/python3.13/dist-packages/ycable/ycable_utilities/y_cable_helper.py", line 21, in <module>
2026 Jun  2 04:07:51.741411 str3-8101c1-07 INFO pmon#supervisord: ycabled     from proto_out import linkmgr_grpc_driver_pb2_grpc
2026 Jun  2 04:07:51.741420 str3-8101c1-07 INFO pmon#supervisord: ycabled   File "/usr/local/lib/python3.13/dist-packages/proto_out/linkmgr_grpc_driver_pb2_grpc.py", line 19, in <module>
2026 Jun  2 04:07:51.741430 str3-8101c1-07 INFO pmon#supervisord: ycabled     raise RuntimeError(
2026 Jun  2 04:07:51.741446 str3-8101c1-07 INFO pmon#supervisord: ycabled     ...<5 lines>...
2026 Jun  2 04:07:51.741460 str3-8101c1-07 INFO pmon#supervisord: ycabled     )
2026 Jun  2 04:07:51.741477 str3-8101c1-07 INFO pmon#supervisord: ycabled RuntimeError: The grpc package installed is at version 1.67.1, but the generated code in proto_out/linkmgr_grpc_driver_pb2_grpc.py depends on grpcio>=1.71.0. Please upgrade your grpc module to grpcio>=1.71.0 or downgrade your generated code using grpcio-tools<=1.67.1

@saiarcot895

Copy link
Copy Markdown
Contributor

Fix for this should be in #27698. Testing is pending.

@xwjiang-ms xwjiang-ms closed this Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants