Conversation
…d SSH resource Adds the complete core infrastructure for Salt Resources (SRN), allowing resources to be targeted and executed against independently from the managing minion, along with a full SSH resource implementation. Targeting layer: - T@ and M@ compound match engines (resource_match, managing_minion_match) - CkMinions._check_resource_minions and _augment_with_resources so that glob targets like '*' automatically include managed resource IDs - Master-side resource registration (_register_resources in AESFuncs) and minion-side _register_resources_with_master on connect Loader layer: - salt.loader.resource(): loads salt/resource/*.py, packs as __resource_funcs__ - salt.loader.resource_modules(opts, resource_type): isolated per-type execution-module loader with opts["resource_type"] injected; one instance per managed type per minion process (not per device) Minion dispatch: - gen_modules() initialises self.resource_funcs and self.resource_loaders - _thread_return routes resource jobs to the per-type loader, injects __resource__ and per-resource __grains__ before each call - Unknown functions for a resource type fail loudly instead of silently executing on the managing minion - _execute_job_function accepts an optional functions= loader parameter - JID deduplication bypassed for resource_job loads (shared parent JID) - _NO_RESOURCE_FUNS prevents internal Salt plumbing from dispatching to resources Dummy resource (example/test implementation): - salt/resource/dummy.py: full dummy resource (ping, grains, services, pkgs) - salt/modules/dummyresource_test.py: Pattern B execution module gating on opts["resource_type"] == "dummy", routes test.ping to dummy.ping() SSH resource: - salt/resource/ssh.py: SSH resource module using salt-ssh Shell transport - salt/modules/sshresource_cmd.py: cmd.* surface for SSH resources - salt/modules/sshresource_pkg.py: pkg.* surface for SSH resources - salt/modules/sshresource_state.py: state.* surface for SSH resources - salt/modules/sshresource_test.py: test.ping for SSH resources Also adds salt/grains/resources.py, design documents, example loader scripts, srv/ pillar configuration, and a Docker environment for testing. Made-with: Cursor
salt.loader.resource_modules was missing "__resource__" from its pack, so LazyLoader never created a NamedLoaderContext for it and modules like sshresource_state raised NameError when accessing __resource__["id"]. Added the sentinel entry so the loader wires __resource__ to resource_ctxvar — the per-thread contextvar set by _thread_return before each resource job executes. Added a regression test (test_resource_modules_packs_resource_dunder) and a full suite of unit and integration tests covering T@ targeting, wildcard augmentation, resource_ctxvar thread isolation, and the _resolve_resource_targets dispatch path. Fixed lint issues across the new test files (blacklisted unittest.mock imports, f-string without interpolation, unused import, broad exception catch).
…rces
Three bugs fixed and one new feature added:
1. salt '*' test.ping was incorrectly dispatching to resources when
targeting a specific minion by name (e.g. salt 'minion' test.ping).
_resolve_resource_targets now only dispatches to resources for wildcard
glob patterns (* ? [). The master-side _augment_with_resources guard
is updated consistently.
2. Removing a resource type from the pillar and running sync_all left
stale resource IDs in the master cache indefinitely, causing phantom
entries in salt '*' output. Three co-operating fixes:
- _discover_resources now distinguishes "no resources key in pillar"
(preserve opts for config-file deployments) from "resources key
present but empty" (authoritative removal — clear opts).
- _register_resources_with_master no longer short-circuits on an empty
resource dict; it always sends the registration so the master cache
is overwritten with the current (possibly empty) state.
- pillar_refresh re-discovers resources and re-registers with the
master after successfully compiling a new pillar, so a sync_all
picks up removals without requiring a minion restart.
3. _augment_with_resources had no error handling; a cache driver failure
would propagate into check_minions and silently discard all PKI-based
minion IDs. Cache errors are now caught, logged, and degraded
gracefully.
New: saltutil.sync_resources / refresh_resources — explicit resource
sync modelled after sync_modules/refresh_modules. sync_all now includes
ret["resources"] and fires a resource_refresh event that triggers
_discover_resources + _register_resources_with_master on the minion.
For state functions (state.apply, state.highstate, state.sls, etc.) run against a wildcard glob, resource jobs are now executed inline by the managing minion and their results folded into a single response block, producing one Summary section instead of one per resource. Key changes: - salt/utils/minions.py: add _MERGE_RESOURCE_FUNS; check_minions accepts fun= and skips resource augmentation for merge-mode functions so the master does not wait for separate per-resource responses - salt/master.py: pass fun= to check_minions at publish time - salt/minion.py: add _MERGE_RESOURCE_FUNS and _prefix_resource_state_key (refactored to @staticmethod); _handle_payload skips separate resource dispatch for merge-mode functions; _thread_return executes resources inline and merges prefixed state keys into the managing minion's ret, reporting Result: False for unsupported functions and missing loaders - salt/modules/sshresource_state.py: highstate() returns a no_|-states state dict (Result: False) when the top file has no match; _exec_state_pkg recovers valid state dicts from SSHCommandExecutionError instead of re-raising and losing the result Tests: - test_minions_resources.py: merge-fun skips augmentation, non-merge and no-fun still augment, compound targets unaffected - test_minion_resources.py: _MERGE_RESOURCE_FUNS membership and sync invariant, _prefix_resource_state_key correctness and fallback, merge block no-loader/unsupported/dict/string branches - test_sshresource_state.py (new): empty-chunks no-top-file return, SSHCommandExecutionError recovery and re-raise cases, normal envelope path
- salt/states/saltutil.py: add sync_resources() state function to match the sync_resources() module function added in saltutil.py; the existing test enforces a 1:1 correspondence between all sync_* module and state functions - tests/pytests/unit/client/test_netapi.py: update test_run_log to use AsyncMock for process_manager.run(), which our netapi.py change wrapped in asyncio.run() and therefore requires a coroutine, not a plain Mock
…on job targeting - salt/state.py: add __pub_resource_targets, __pub_minion_is_target, __pub_resource_target, and __pub_resource_job to STATE_REQUISITE_KEYWORDS so they are treated as internal pub metadata and stripped before being passed to state module functions (e.g. git.present, ssh_known_hosts.present) - salt/utils/minions.py: guard the _MERGE_RESOURCE_FUNS membership check with isinstance(fun, str) — multifunction jobs pass fun as a list which is not hashable, causing a TypeError swallowed by the broad except that returned an empty minion list and "No minions matched the target" - tests: add test_check_minions_list_fun_still_augments covering the list fun case, and CI-TRACKING.md to record known failures and regression tests
The salt-ssh relenv work merged into master introduced three changes that required corresponding updates in our SSH resource module: 1. Single.cmd_block() now calls mod_data(self.fsclient) unconditionally to regenerate extension modules before every remote execution. Passing fsclient=None (the default) crashes immediately at fsclient.opts. _exec_state_pkg() now creates a fresh FSClient from the cached master opts and passes it via the fsclient= kwarg to Single. 2. Single.__init__ renames thin_dir from the default *_salt suffix to *_salt_relenv when relenv=True, then writes the updated path back into opts["thin_dir"]. The previous code built the state.pkg argv string before constructing Single, so the baked-in path diverged from where the state tar was actually sent. The argv is now set after __init__ so both the send path and the command string use the finalized thin_dir. 3. _relenv_path() hardcoded linux/x86_64, which would silently deploy the wrong tarball to arm64 hosts. It now probes x86_64 then arm64 and returns the first path that exists locally, or None. A None return tells Single.__init__ to call detect_os_arch() itself — one extra SSH round-trip as a fallback, but always correct. Also removes the leftover debug echo from SSH_SH_SHIM_RELENV that was writing SALT_CALL_CMD diagnostics to stderr on every remote execution. Tests added: TestRelenvPath (4 cases), fsclient mock added to all _exec_state_pkg test helpers, and test_single_receives_fsclient verifies the kwarg is threaded through.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
What issues does this PR fix or reference?
Fixes #68888