Skip to content

Async batch#60269

Open
vzhestkov wants to merge 11 commits into
saltstack:masterfrom
vzhestkov:batch-async-impl
Open

Async batch#60269
vzhestkov wants to merge 11 commits into
saltstack:masterfrom
vzhestkov:batch-async-impl

Conversation

@vzhestkov
Copy link
Copy Markdown
Contributor

What does this PR do?

Implementation of RFC#0002

#50546 port to master

All related tests migrated to pytest

@vzhestkov vzhestkov requested a review from a team as a code owner May 28, 2021 06:32
@vzhestkov vzhestkov requested review from twangboy and removed request for a team May 28, 2021 06:32
@garethgreenaway
Copy link
Copy Markdown
Contributor

@vzhestkov Just a heads up, there are quite a few merge conflicts here.

@vzhestkov
Copy link
Copy Markdown
Contributor Author

@garethgreenaway thanks for pinging, I'll resolve them once I get some time for it, will try to do it as soon as possible

@garethgreenaway garethgreenaway added the Phosphorus v3005.0 Release code name and version label Apr 20, 2022
Comment thread salt/cli/batch.py Outdated
Comment thread salt/cli/batch.py Outdated
Comment thread salt/cli/batch.py Outdated
Comment thread salt/cli/batch.py Outdated
Comment thread salt/cli/batch_async.py Outdated
Comment thread salt/cli/batch_async.py Outdated
Comment thread salt/cli/batch_async.py Outdated
Comment thread salt/master.py Outdated
Comment thread salt/utils/event.py Outdated
Comment thread tests/pytests/unit/cli/test_batch.py Outdated
@Ch3LL
Copy link
Copy Markdown
Contributor

Ch3LL commented May 20, 2022

There are test failures

@garethgreenaway
Copy link
Copy Markdown
Contributor

@vzhestkov A few test failures here and some requested changes.

@Ch3LL
Copy link
Copy Markdown
Contributor

Ch3LL commented Jun 9, 2022

We are close to tagging for the RC release for phosphorus. I would like to get this one in if the tests get fixed up

@vzhestkov
Copy link
Copy Markdown
Contributor Author

Thanks for pinging. I'll try to fix all the suggestions today or tomorrow morning

@vzhestkov vzhestkov force-pushed the batch-async-impl branch 9 times, most recently from 70c4853 to e1e4b03 Compare June 10, 2022 14:32
Comment thread salt/cli/batch_async.py Outdated
Comment thread salt/cli/batch_async.py Outdated
Comment thread salt/master.py Outdated
Comment thread salt/master.py Outdated
Comment thread salt/utils/event.py Outdated
@vzhestkov
Copy link
Copy Markdown
Contributor Author

@twangboy thanks for pointing again, accidentally missed it. Fixed

twangboy
twangboy previously approved these changes Jun 14, 2022
@Ch3LL
Copy link
Copy Markdown
Contributor

Ch3LL commented Jun 16, 2022

Theres a few tests failing on multiple platforms that seem to be related.

@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 08:44 — with GitHub Actions Inactive
@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 08:44 — with GitHub Actions Inactive
@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 08:44 — with GitHub Actions Inactive
@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 09:00 — with GitHub Actions Inactive
@vzhestkov vzhestkov force-pushed the batch-async-impl branch 2 times, most recently from 40b7e09 to 6869add Compare August 7, 2023 09:40
@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 09:55 — with GitHub Actions Inactive
@vzhestkov vzhestkov temporarily deployed to ci August 7, 2023 09:55 — with GitHub Actions Inactive
twangboy
twangboy previously approved these changes Feb 26, 2024
twangboy
twangboy previously approved these changes Jan 29, 2025
@twangboy
Copy link
Copy Markdown
Contributor

twangboy commented Aug 7, 2025

Looks like there are some test failures here

@twangboy
Copy link
Copy Markdown
Contributor

Please fix the conflicts

@dwoz
Copy link
Copy Markdown
Contributor

dwoz commented Mar 17, 2026

@vzhestkov Can you please rebase this 🙏 . We'll try to get it into 3008.0

@whytewolf
Copy link
Copy Markdown
Collaborator

looks like the current errors are related to the changes.

@vzhestkov
Copy link
Copy Markdown
Contributor Author

@whytewolf sure, I will check.

vzhestkov and others added 11 commits May 21, 2026 16:44
Co-authored-by: Mihai Dinca <dincamihai@gmail.com>
batch async: enhance trace logging

batch async: do not block event handler

additionally, only try to execute next batch once all previous minions
has been processed somehow

batch async: do not block current find_job execution

batch async: just start batch if all minions returns the ping

batch_async: ensure the start/done events are fired

batch async: do not wait to finalize batch execution

batch async: only trigger find job for current batch active minions

Prevent traceback due missing 'send' method when socket was closed

Modify some batch_async trace logs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants