Skip to content

Commit 4511c05

Browse files
committed
feat(datafusion): organize Cargo.toml and clarify IPCStreamWriter codec contract
- Cleaned up alphabetical ordering in Cargo.toml. - Added comments to clarify feature-only dependencies related to IPCStreamWriter in spill/mod.rs. - Documented codec contract requirement in IPCStreamWriter::new for better understanding of feature dependencies.
1 parent 3aefba7 commit 4511c05

3 files changed

Lines changed: 18 additions & 0 deletions

File tree

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

datafusion/physical-plan/Cargo.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@ name = "datafusion_physical_plan"
4949
[dependencies]
5050
arrow = { workspace = true }
5151
arrow-data = { workspace = true }
52+
# Feature-only dep (arrow re-exports this crate as `arrow::ipc`): lz4 and zstd
53+
# codec support required by IPCStreamWriter in spill/mod.rs. These features must
54+
# stay in sync with the SpillCompression variants in datafusion-common. Declaring
55+
# the dependency here makes the contract explicit and local to the crate that owns
56+
# spill, preventing silent regressions if workspace-level arrow-ipc features are
57+
# narrowed (see commit 14c18ece4 for a prior regression caused by this coupling).
58+
arrow-ipc = { workspace = true, features = ["lz4", "zstd"] }
5259
arrow-ord = { workspace = true }
5360
arrow-schema = { workspace = true }
5461
async-trait = { workspace = true }

datafusion/physical-plan/src/spill/mod.rs

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,16 @@ struct IPCStreamWriter {
290290

291291
impl IPCStreamWriter {
292292
/// Create new writer
293+
///
294+
/// # Codec contract
295+
///
296+
/// `arrow-ipc` must be compiled with the `lz4` and `zstd` features
297+
/// (declared explicitly in `datafusion-physical-plan/Cargo.toml`). If
298+
/// those features are absent, `try_with_compression` will return an
299+
/// error at runtime for [`SpillCompression::Lz4Frame`] and
300+
/// [`SpillCompression::Zstd`] variants. The Cargo dependency keeps this
301+
/// contract local and compiler-visible rather than relying solely on
302+
/// workspace-level feature unification.
293303
pub fn new(
294304
path: &Path,
295305
schema: &Schema,

0 commit comments

Comments
 (0)