feat(onpair): promote OnPair to a stable read-only encoding#8872
Merged
Conversation
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Move encodings/experimental/onpair -> encodings/onpair and register the OnPair reader unconditionally. The writer stays gated behind the `unstable_encodings` feature (file ALLOWED_ENCODINGS and btrblocks ALL_SCHEMES/scheme module), so files with OnPair can be read but not yet written by default. Signed-off-by: Francesco Gargiulo <gargiulo.fr@gmail.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Polar Signals Profiling ResultsLatest Run
Previous Runs (2)
Powered by Polar Signals Cloud |
Merging this PR will not alter performance
Comparing Footnotes
|
Contributor
Benchmarks: Vortex queries 📖Verdict: No clear signal (low confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.067x ➖, 0↑ 0↓)
datafusion / parquet (0.996x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.973x ➖, 0↑ 0↓)
duckdb / parquet (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
gargiulofrancesco
marked this pull request as ready for review
July 21, 2026 10:00
joseph-isaacs
approved these changes
Jul 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
OnPair has been living under
encodings/experimental/while it stabilized. We nowwant OnPair-encoded files to be readable by default, without yet committing to writing
them. This PR promotes OnPair to a stable encoding but enables it on the read path
only — the compressor/writer stays gated behind the
unstable_encodingsfeature sowe can turn on writing in a follow-up.
What changes are included in this PR?
encodings/experimental/onpair→encodings/onpair(100% filerenames) and drop it from the "Experimental encodings" group in the workspace
Cargo.toml.register_default_encodings, so anysession can decode OnPair arrays and push down filters against them.
ALLOWED_ENCODINGS(file write strategy) only inserts OnPair under#[cfg(feature = "unstable_encodings")].ALL_SCHEMESand theonpairscheme module are gated behindunstable_encodings, so the default compressor never selects OnPair.default_writer_does_not_allow_onpairasserts OnPair is absent from the defaultwrite allowlist; the reader-registration test asserts OnPair is registered with a
Filter execute-parent kernel.
unstable_encodingsand renamed to make the scope explicit(
test_unstable_all_schemes_includes_onpair,test_unstable_default_btrblocks_compressor_selects_onpair).Verified locally:
vortex-onpair,vortex-btrblocks,vortex-file—169 tests passed.
unstable_encodings(writer-enabled) config: 176 tests passed (the extra 7 are thefeature-gated write-path tests).
cargo clippy --all-targets --all-features(affected crates) clean;cargo +nightly fmt --allclean;git diff --checkclean.What APIs are changed? Are there any user-facing changes?
vortex-onpaircrate moves toencodings/onpair(path only; crate name andversion are unchanged).
Writing OnPair still requires the
unstable_encodingsfeature — no default writebehavior changes.