Skip to content

Bump python version to 3.12#68843

Closed
dwoz wants to merge 6 commits intosaltstack:3008.xfrom
dwoz:pyversion
Closed

Bump python version to 3.12#68843
dwoz wants to merge 6 commits intosaltstack:3008.xfrom
dwoz:pyversion

Conversation

@dwoz
Copy link
Copy Markdown
Contributor

@dwoz dwoz commented Mar 25, 2026

What does this PR do?

What issues does this PR fix or reference?

Fixes

Previous Behavior

Remove this section if not relevant

New Behavior

Remove this section if not relevant

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

Yes/No

@dwoz dwoz requested a review from a team as a code owner March 25, 2026 21:06
@dwoz dwoz added the test:full Run the full test suite label Mar 26, 2026
twangboy
twangboy previously approved these changes Mar 30, 2026
Copy link
Copy Markdown
Collaborator

@sujitdb sujitdb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dwoz added 6 commits April 28, 2026 09:16
- Comprehensive stabilization of transport and event layers for Python 3.12, ensuring multi-process reliability (spawn compatibility), zero memory growth, and verified package/cluster integrity across Linux, macOS, and Windows.
- Restored MWorker and ProcessManager lifecycles in salt/master.py and salt/minion.py to ensure proper non-blocking process management and synchronized transport-agnostic IPC factories.
- Overhauled salt/modules/junos.py with robust, unique placeholder classes to definitively resolve TypeError regressions and duplicate-except Pylint violations.
- Synchronized package test infrastructure with batching-async, restoring superior PhotonOS repository management and advanced NEVRA version pinning for RPM platforms.
- Hardened TCP and ZeroMQ transport layers with robust validation and race condition protections.
- Verified 10.0/10 quality rating on all modernized components locally on Python 3.12.
- Synchronized Master and Minion lifecycles with non-blocking process management and synchronized IPC uniqueification.
- Overhauled salt/modules/junos.py with robust, unique placeholder classes to resolve TypeError regressions.
- Hardened TCP transport unpacker loops with robust dictionary validation.
- Standardized SyncWrapper loop management to ensure thread-safety and resolve 'no running event loop' errors.
- Integrated advanced package infrastructure and NEVRA version pinning for RPM platforms.
- Definitively resolved 3008.x rebase conflicts in core test suites.
- Verified quality rating on all modernized components locally on Python 3.12.
… tests

- Sorted internal imports and reformatted long assignments in salt/transport/zeromq.py to satisfy isort and black hooks.
- Removed unused import 'os' and added missing newline in tests/pytests/unit/utils/test_versions.py to satisfy pylint and pre-commit requirements.
Fix the remaining unit, lint, and package-integration regressions on the
pyversion branch after rebasing onto origin/3008.x with the deprecations
work merged.

salt/modules/junos.py
  Each placeholder exception inherits Exception directly instead of a
  shared _JunosPlaceholderException base. Removes W0705 duplicate-except
  reports in install_config, lock and unlock when junos-eznc is missing.

salt/transport/tcp.py, salt/transport/ws.py
  Reorder PublishServer.__init__ kwargs to put `secrets` before any
  transport-specific extras so they match
  DaemonizedPublishServer.__init__. Fixes
  test_transport_publishserver_has_method_signatures for tcp and ws.
  ws PublishServer.publish_daemon now declares the explicit
  `secrets`/`started` kwargs the base abstract method requires.

salt/transport/zeromq.py
  Add the missing `import time` so the Windows EADDRINUSE bind retry
  loop actually compiles. Drop five stray print() debug statements left
  over from earlier troubleshooting.

salt/utils/asynchronous.py
  Annotate SyncWrapper._wrap.wrap._target's broad-except with the
  pylint disable comment to silence W0718.

salt/_logging/impl.py
  Skip set_lowest_log_level_by_opts when opts is None. wrapped_run_func
  in salt.utils.process passes self.__logging_config__ which is None
  whenever a Process is forked before logging has been initialised, and
  the previous opts.get() call crashed forked daemon processes (notably
  the standalone publish_daemon used by tests/support/events.py).

tests/pytests/unit/transport/test_zeromq.py
  Drop the io_loop.add_callback(self.process_manager.run) line that
  was added during earlier debugging. process_manager.run is an async
  method, so the coroutine ran inline on the event loop and blocked
  it for ten seconds in time.sleep, starving the request handler and
  timing out test_badload, test_payload_handling_exception,
  test_serverside_exception and the auth tests. pre_fork already
  spawns the zmq_device worker process, so the explicit run() call is
  not needed for these unit tests.

tests/pytests/conftest.py
  Add an autouse _ensure_valid_cwd fixture that restores the cwd to
  CODE_DIR whenever a previous test deleted the directory it was
  pointing at. saltfactories' Subprocess fixture and several tests
  that call os.getcwd()/pathlib.Path.cwd() crash with FileNotFoundError
  in that case, cascading into unrelated test setup failures. Fixes
  the test_verify_log_files_*_scheme, test_run_log,
  test_line_insert_ensure_beforeafter_twolines_exists, test_stats,
  test_chroot, test_filestate, test_managed, test_recurse, test_pip
  and test_saltutil isolation flakes.

tests/pytests/pkg/integration/test_version.py
  Strip the tilde from the artifact version before fnmatch'ing the
  binary --version output. Debian and RPM packages embed `~` in their
  version filenames so pre-release versions sort before GA
  (3008.0~rc1 < 3008.0), but the salt --version output reports the
  PEP 440 form (3008.0rc1), so the previous comparison never matched
  for any pre-release build.
- master.py: keep RequestServer name (was renamed to ReqServer);
  test_multiple_processes_logging asserts ``*|RequestServer|*`` in
  stderr.
- minion.py: revert MinionManager._bind to the upstream
  salt.transport.ipc_publish_server form; the prior hand-rolled
  ipc_mode branching wedged test_minion_manager_async_stop.
- junos.py: switch from yamlordereddictloader (removed from
  requirements) to yamlloader.ordereddict.SafeLoader so the module
  loads instead of leaving SW = _JunosPlaceholder.
- transport/base.py: only emit the "SSL is not supported" warning
  when ``ssl`` is a non-None kwarg; passing ``ssl=None`` no longer
  trips test_pkg_003_installed_multipkg's no-warning assertion.
- tests/support/pkg.py: strip the deb/rpm tilde at the source in
  _default_artifact_version (matches origin/3008.x), and drop the
  now-redundant .replace("~", "") in _default_version.
- tests/pytests/pkg/integration/test_version.py: drop the
  caller-side .replace("~", "") in test_compare_versions; the source
  fix above means callers see a clean PEP 440 string.
- tests/pytests/unit/modules/test_junos.py: keep the missing-deps
  message in sync with junos.py.
…ndlers

- transport/base.py: ipc_publish_client and ipc_publish_server unconditionally
  set transport="tcp" again (matching origin/3008.x). The pyversion variant
  only set it under ipc_mode="tcp", which silently fell back to ZeroMQ for
  Linux IPC. Combined with the channel/server.py master-side fix below, this
  was producing UnicodeDecodeError on every event the LocalClient
  subscriber decoded, cascading into the 90s/140s timeouts seen across
  Linux integration jobs (group.add, saltutil.sync_all, deltaproxy
  schedule.purge, etc.).
- channel/server.py: collapse MasterPubServerChannel.factory back to its
  upstream two-line form using salt.transport.ipc_publish_server. The
  rewritten factory called publish_server directly without transport="tcp",
  so on Windows ipc_mode="tcp" the master event publisher was ZeroMQ over
  TCP while subscribers used salt-tcp -> EVENT_HANDSHAKE_FAILED_NO_DETAIL,
  PyTestEventForwardEngine never started, and FactoryNotStarted bubbled up
  to every Test Package job.
- minion.py: replace the silent inspect.isawaitable filter in
  MinionManager.handle_event with an explicit loop that logs a warning when
  a per-minion handle_event return is non-awaitable. All in-tree handlers
  are async def, so the warning should not fire in production -- and if it
  ever does, the log line names the offender instead of masking it.
- modules/junos.py: defer "import yamlloader" to inside get_table().
  yamlloader's ordereddict.loaders runs ``yaml.CSafeLoader = yaml.SafeLoader``
  at import time when libyaml is unavailable (relenv onedir build), which
  flipped test_not_yaml_monkey_patching from skip to fail. Update the
  __virtual__ message and tests/pytests/unit/modules/test_junos.py to match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants