Skip to content

feat: add files.unarchive operation#1631

Merged
Fizzadar merged 5 commits intopyinfra-dev:3.xfrom
KalvadTech:feat/files-unarchive
Apr 21, 2026
Merged

feat: add files.unarchive operation#1631
Fizzadar merged 5 commits intopyinfra-dev:3.xfrom
KalvadTech:feat/files-unarchive

Conversation

@wowi42
Copy link
Copy Markdown
Collaborator

@wowi42 wowi42 commented Mar 24, 2026

Extract archive files on remote systems. Supports tar (.tar, .tar.gz, .tar.bz2, .tar.xz, .tar.zst) and zip formats. Archives can be local (uploaded automatically) or already on the remote host. Includes idempotency via creates parameter and optional chown support.

Closes #1546

  • Pull request is based on the default branch (3.x at this time)
  • Pull request includes tests for any new/updated operations/facts
  • Pull request includes documentation for any new/updated operations/facts
  • Tests pass (see scripts/dev-test.sh)
  • Type checking & code style passes (see scripts/dev-lint.sh)

wowi42 added 3 commits March 24, 2026 15:11
Extract archive files on remote systems. Supports tar (.tar, .tar.gz,
.tar.bz2, .tar.xz, .tar.zst) and zip formats. Archives can be local
(uploaded automatically) or already on the remote host. Includes
idempotency via `creates` parameter and optional chown support.

Closes pyinfra-dev#1546
Pass QuoteString args directly to StringCommand instead of extending
a list[str] with them.
Copy link
Copy Markdown
Member

@Fizzadar Fizzadar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of notes!

Comment thread tests/operations/files.unarchive/extract_with_extra_opts.json Outdated
Comment thread src/pyinfra/operations/files.py Outdated
wowi42 added 2 commits April 21, 2026 07:46
…use file_utils.chown

- Split -f from the tar format flags so extra_opts can no longer be
  mistaken for the archive path. Command shape is now
  'tar <flags> <extras> -f <archive> -C <dest>'.
- Delegate ownership to file_utils.chown instead of a hand-rolled
  StringCommand, matching the rest of the files operations.
- Add fixtures for .tar.zst (multi-flag format) and unzip with extra_opts.
@wowi42 wowi42 requested a review from Fizzadar April 21, 2026 05:48
@wowi42
Copy link
Copy Markdown
Collaborator Author

wowi42 commented Apr 21, 2026

@Fizzadar both review points addressed in 0774e05:

  1. tar -f ordering: split -f out of the format flags so extra_opts can't shadow the archive path. Command shape is now tar <flags> <extras> -f <archive> -C <dest>. Added a .tar.zst fixture to guard the multi-flag case.
  2. chown: switched to file_utils.chown(dest, user, group, recursive=True).

End-to-end run across four hosts (different OSes), covering upload-and-extract, creates idempotency, extra_opts, and chown:

test_unarchive.py

$ pyinfra inventory.py test_unarchive.py
--> Loading config...
--> Loading inventory...
--> Connecting to hosts...
    [10.0.10.23] Connected
    [10.0.10.20] Connected
    [10.0.10.21] Connected
    [10.0.10.22] Connected

--> Preparing operation files...
    Loading: test_unarchive.py
    [10.0.10.20] Ready: test_unarchive.py
    [10.0.10.23] Ready: test_unarchive.py
    [10.0.10.21] Ready: test_unarchive.py
    [10.0.10.22] Ready: test_unarchive.py

--> Detected changes:
    Operation                                 Change                                               Conditional Change
    Clean previous extraction roots           4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Upload and extract tar.gz                 4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Extract with creates marker               4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Re-extract should skip (creates exists)   4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Extract with extra_opts (verbose)         4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Extract and chown tree                    4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Verify extracted payloads                 4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Verify chown applied                      4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -
    Tear down extraction roots                4 (10.0.10.20, 10.0.10.21, 10.0.10.22, 10.0.10.23)   -

    Detected changes may not include every change pyinfra will execute.
    Hidden side effects of operations may alter behaviour of future operations,
    this will be shown in the results. The remote state will always be updated
    to reflect the state defined by the input operations.

    Detected changes displayed above, skip this step with -y

--> Beginning operation run...
--> Starting operation: Clean previous extraction roots
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Upload and extract tar.gz
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Extract with creates marker
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Re-extract should skip (creates exists)
    [10.0.10.20] No changes
    [10.0.10.23] No changes
    [10.0.10.21] No changes
    [10.0.10.22] No changes

--> Starting operation: Extract with extra_opts (verbose)
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Extract and chown tree
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Verify extracted payloads
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Starting operation: Verify chown applied
    [10.0.10.20] Success
    [10.0.10.22] Success
    [10.0.10.21] Success
    [10.0.10.23] Success

--> Starting operation: Tear down extraction roots
    [10.0.10.20] Success
    [10.0.10.21] Success
    [10.0.10.23] Success
    [10.0.10.22] Success

--> Results:
    Operation                                 Hosts   Success   Error   No Change
    Clean previous extraction roots           4       4         -       -
    Upload and extract tar.gz                 4       4         -       -
    Extract with creates marker               4       4         -       -
    Re-extract should skip (creates exists)   4       -         -       4
    Extract with extra_opts (verbose)         4       4         -       -
    Extract and chown tree                    4       4         -       -
    Verify extracted payloads                 4       4         -       -
    Verify chown applied                      4       4         -       -
    Tear down extraction roots                4       4         -       -
    Grand total                               36      32        -       4

--> Disconnecting from hosts...

The Re-extract should skip row shows creates idempotency working (4 no-change). Everything else is green.

Copy link
Copy Markdown
Member

@Fizzadar Fizzadar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, looks good to me!

@Fizzadar Fizzadar merged commit c11fa48 into pyinfra-dev:3.x Apr 21, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add files.unarchive + files.archive operations

2 participants