Build zstd with multithreading enabled#762
Merged
Merged
Conversation
Member
Author
|
We can add a test case that verifies this at runtime. |
|
This does not solve #761 (I'm having the same timings as on the issue), however it is a good thing that should be done anyways. I can confirm that this PR adds support for multithreading. For example, without this PR, the following exception would be raised: >>> from compression import zstd
>>> zstd.compress(b'foo', options={zstd.CompressionParameter.nb_workers: 2})
Traceback (most recent call last):
File "<python-input-1>", line 1, in <module>
zstd.compress(b'foo', options={zstd.CompressionParameter.nb_workers: 2})
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/redacted/.local/share/uv/python/cpython-3.14.0rc1-linux-x86_64-gnu/lib/python3.14/compression/zstd/__init__.py", line 148, in compress
comp = ZstdCompressor(level=level, options=options, zstd_dict=zstd_dict)
ValueError: compression parameter 'nb_workers' received an illegal value 2; the valid range is [0, 0]Now with this PR (using >>> from compression import zstd
>>> zstd.compress(b'foo', options={zstd.CompressionParameter.nb_workers: 2})
b'(\xb5/\xfd \x03\x19\x00\x00foo' |
charliermarsh
approved these changes
Aug 18, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I verified this on macOS
See #761
I'm not sure if this is all of the performance difference, but using threads should make a significant difference. #761 (comment) notes that threads are not used by default, so I think there's more going on — but we should enable this at build time regardless. See also #768.