fix: preserve members order in ExtractorProcessor.__call__()#537
Open
terminalchai wants to merge 1 commit into
Open
fix: preserve members order in ExtractorProcessor.__call__()#537terminalchai wants to merge 1 commit into
terminalchai wants to merge 1 commit into
Conversation
When the 'members' argument is supplied to Unzip or Untar, the returned list
of extracted file paths now follows the same order as 'members' instead of
the arbitrary order produced by os.walk().
Previously the code walked the extract directory and appended files whenever
any member prefix matched, meaning the order of results was determined by
the filesystem rather than the caller.
The fix replaces the single os.walk() loop with two code paths:
- members is None -> walk and collect all files (behaviour unchanged)
- members provided -> walk once into a dict {member: full_path}, then
return [dict[m] for m in self.members] to guarantee caller-specified order.
Walking only once preserves O(N) complexity.
Add test_unpacking_members_order_preserved for both Unzip and Untar.
Closes fatiando#457
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #457.
When \members\ is supplied to \Unzip\ or \Untar, the returned list of file paths now follows the *same order as \members* instead of the arbitrary order produced by \os.walk().
Root cause
The previous implementation iterated \os.walk(extract_dir)\ and appended a file path whenever any member prefix matched. The walk order is filesystem- and OS-dependent, making the resulting list order unpredictable and unrelated to the caller-specified \members.
Fix
Replaced the single \os.walk\ loop with two code paths:
Walking the directory only once preserves the existing O(N) complexity (N = number of extracted files), as suggested by @santisoler in the issue.
Changes
Test
34 passed, 7 deselected in 25.30sBoth \Unzip\ and \Untar\ variants of the new test confirm that requesting members in forward then reverse order produces correspondingly ordered results.