Skip to content

docs: introduce Vortex editions for users of the file format#8852

Closed
joseph-isaacs wants to merge 1 commit into
developfrom
claude/editions-design-doc-fiicrg
Closed

docs: introduce Vortex editions for users of the file format#8852
joseph-isaacs wants to merge 1 commit into
developfrom
claude/editions-design-doc-fiicrg

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

Vortex needs a named, user-visible compatibility contract for which encodings a writer may put in a file. This PR introduces editions: a named grouping of encodings that records exactly which encodings a Vortex file may contain and when each encoding joined the set. Writing under an edition guarantees every future Vortex release can read — and execute queries over — the file. The concept fills the forward/backward-compatibility seam already noted in register_default_encodings ("different Vortex 'Editions' that may support different sets of encodings") and gives unknown-encoding errors a stable docs page to link to. A follow-up PR adds the Rust implementation (vortex-edition crate) and the generated registry.

What changes are included in this PR?

  • docs/specs/editions.md — a user-facing spec page, structured around what readers need: the definition (edition = which encodings, added when), a step-by-step "resolving an unknown-encoding error" section, writing with an edition (default just works; pin older editions; additive families; explicit opt-out), how editions change (frozen sets, drafts, deprecation never breaks reads, edition ≠ file-format version), and the registry.
  • Defines the first edition core2026.07.0 — exactly the 32 encodings the default file writer emits today (derived from ALLOWED_ENCODINGS in vortex-file, including vortex.zstd since zstd is a default feature) — and the core2026.10.0 draft as the staging area for the next edition. Experimental encodings (vortex.onpair, vortex.zstd_buffers, vortex.patched, vortex.piecewise-sequence) are explicitly excluded.
  • Adds the page to the specs toctree and cross-links it from the file-format spec.

Docs-only change; validated with a clean uv run --all-packages make -C docs html build.

What APIs are changed? Are there any user-facing changes?

No API changes. New user-facing documentation page at docs.vortex.dev/specs/editions.html.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KdR42Svu74NcNzC7XJYwLr

An edition names the exact set of encodings a writer may put in a file,
giving every file written under it a forever read-and-execute guarantee.
This spec page explains the model for users of Vortex: editions
constrain writers while readers are cumulative, family-namespaced
editions (core2026.07.0) compose additively, writer presets choose
within an edition without affecting portability, and unknown-encoding
errors link back to the registry. Defines the first edition
core2026.07.0 (the 32 encodings the default writer emits today) and the
core2026.10.0 draft.

Signed-off-by: "Claude" <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KdR42Svu74NcNzC7XJYwLr
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done e25d1a2 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries 📖

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +2.7%
Engines: DataFusion No clear signal (+6.2%, low confidence) · DuckDB No clear signal (-0.8%, low confidence)
Vortex (geomean): 1.006x ➖
Parquet (geomean): 0.993x ➖
Shifts: Parquet (control) -0.7% · Median polish +0.6%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.036x ➖, 0↑ 0↓)
name PR e25d1a2 (ns) base c5e075a (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 9677194 9222307 1.05
vortex_q01/datafusion:vortex-file-compressed 6145791 6003507 1.02
datafusion / parquet (0.976x ➖, 0↑ 0↓)
name PR e25d1a2 (ns) base c5e075a (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 19782544 20168593 0.98
vortex_q01/datafusion:parquet 4449462 4585156 0.97
duckdb / vortex-file-compressed (1.002x ➖, 0↑ 0↓)
name PR e25d1a2 (ns) base c5e075a (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 9920204 9919645 1.00
vortex_q01/duckdb:vortex-file-compressed 5904317 5876722 1.00
duckdb / parquet (1.010x ➖, 0↑ 0↓)
name PR e25d1a2 (ns) base c5e075a (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23389838 23121933 1.01
vortex_q01/duckdb:parquet 9377971 9295209 1.01

No file size changes detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants