feat: Add lz4 compression to array_encodings

[angela-ko]

Relevant issue or PR

To be done prior to implementing in pasteur-types

Changes are basically identical to the changes here
https://github.com/pasteurlabs/pasteur-types/pull/358/changes

Following the design doc here - chose to start with lz4 as the minimal dependency option for compression, and we can add in more optional compression types once it's working
https://pasteurisi.atlassian.net/wiki/spaces/~71202060d9f9d7be6c427dafac7d77e930e293/pages/1191247903/Compression+-+Design+Options

Description of changes

• Add optional dependency for lz4
• Add compress/decompress to array_encodings and output_to_bytes
• Updated cli and tesseract.py to support compression as well

Testing done

Unit testing

[angela-ko]

@dionhaefner @nmheim Let me know if this is what you meant by testing compression in tesseract?

[dionhaefner]

That's a good start, thanks @angela-ko ! As next step, please add minimal, meaningful end-to-end tests that cover this functionality - which I expect are going to fail because I do see some issues with how the new lz4 dependency is added :)

Once everything is passing end-to-end I'll have a closer look at the design choices here.

[dionhaefner]

And please outline your rationale for choosing lz4 specifically as part of the PR body.

[codecov]

Codecov Report

❌ Patch coverage is 75.00000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.04%. Comparing base (f488e85) to head (50ee165).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
tesseract_core/runtime/array_encoding.py	68.29%	10 Missing and 3 partials ⚠️
tesseract_core/runtime/cli.py	80.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #579      +/-   ##
==========================================
- Coverage   77.12%   77.04%   -0.08%     
==========================================
  Files          32       32              
  Lines        4489     4540      +51     
  Branches      738      751      +13     
==========================================
+ Hits         3462     3498      +36     
- Misses        724      735      +11     
- Partials      303      307       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PasteurBot]

Benchmark Results

Benchmarks use a no-op Tesseract to measure pure framework overhead.

🚀 1 faster, ⚠️ 0 slower, ✅ 35 unchanged

Notable changes

Benchmark	Baseline	Current	Change	Status
decoding/binref_10,000,000	8.671ms	7.355ms	-15.2%	🚀 faster

Full results

Benchmark	Baseline	Current	Change	Status
api/apply_1,000	0.507ms	0.496ms	-2.1%	✅
api/apply_100,000	0.511ms	0.500ms	-2.1%	✅
api/apply_10,000,000	0.512ms	0.501ms	-2.1%	✅
cli/apply_1,000	1741.825ms	1674.395ms	-3.9%	✅
cli/apply_100,000	1712.345ms	1646.281ms	-3.9%	✅
cli/apply_10,000,000	1766.230ms	1698.683ms	-3.8%	✅
decoding/base64_1,000	0.028ms	0.029ms	+1.6%	✅
decoding/base64_100,000	0.675ms	0.611ms	-9.5%	✅
decoding/base64_10,000,000	70.940ms	69.295ms	-2.3%	✅
decoding/binref_1,000	0.191ms	0.185ms	-3.2%	✅
decoding/binref_100,000	0.241ms	0.237ms	-1.8%	✅
decoding/binref_10,000,000	8.671ms	7.355ms	-15.2%	🚀 faster
decoding/json_1,000	0.099ms	0.099ms	-0.1%	✅
decoding/json_100,000	9.381ms	9.541ms	+1.7%	✅
decoding/json_10,000,000	1125.177ms	1114.803ms	-0.9%	✅
encoding/base64_1,000	0.028ms	0.027ms	-0.7%	✅
encoding/base64_100,000	0.143ms	0.143ms	-0.1%	✅
encoding/base64_10,000,000	21.645ms	21.476ms	-0.8%	✅
encoding/binref_1,000	0.291ms	0.275ms	-5.5%	✅
encoding/binref_100,000	0.461ms	0.439ms	-4.7%	✅
encoding/binref_10,000,000	16.886ms	16.770ms	-0.7%	✅
encoding/json_1,000	0.143ms	0.143ms	+0.2%	✅
encoding/json_100,000	14.547ms	14.112ms	-3.0%	✅
encoding/json_10,000,000	1524.621ms	1536.751ms	+0.8%	✅
http/apply_1,000	2.968ms	2.921ms	-1.6%	✅
http/apply_100,000	9.047ms	8.259ms	-8.7%	✅
http/apply_10,000,000	664.571ms	662.448ms	-0.3%	✅
roundtrip/base64_1,000	0.059ms	0.058ms	-0.9%	✅
roundtrip/base64_100,000	0.792ms	0.762ms	-3.9%	✅
roundtrip/base64_10,000,000	92.126ms	90.788ms	-1.5%	✅
roundtrip/binref_1,000	0.495ms	0.475ms	-4.1%	✅
roundtrip/binref_100,000	0.708ms	0.684ms	-3.4%	✅
roundtrip/binref_10,000,000	25.391ms	24.987ms	-1.6%	✅
roundtrip/json_1,000	0.248ms	0.247ms	-0.1%	✅
roundtrip/json_100,000	21.721ms	21.647ms	-0.3%	✅
roundtrip/json_10,000,000	2630.327ms	2631.557ms	+0.0%	✅

• Runner: Linux 6.17.0-1013-azure x86_64