Skip to content

Commit b9163a8

Browse files
committed
Add splitfdstream crate for binary streaming format
Add a new crate implementing the splitfdstream binary format, which enables streaming tar archives with external file references. The format uses an 8-byte LE signed prefix followed by optional data: - Negative prefix: inline data of size |prefix| - Non-negative prefix: external file descriptor reference by index Key types: - SplitfdstreamWriter: Build splitfdstreams with inline/external chunks - SplitfdstreamReader: Parse splitfdstreams back into chunks - SplitfdstreamTarReader: Read adapter that reconstructs byte streams This enables zero-copy tar reconstruction from containers-storage's tar-split metadata by streaming headers inline while referencing file content via fd. Assisted-by: OpenCode (Opus 4.5) Signed-off-by: Colin Walters <walters@verbum.org>
1 parent 984ce03 commit b9163a8

2 files changed

Lines changed: 1339 additions & 0 deletions

File tree

crates/splitfdstream/Cargo.toml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
[package]
2+
name = "splitfdstream"
3+
description = "Binary format for serializing data with external file descriptor references"
4+
keywords = ["splitfdstream", "fd", "serialization"]
5+
6+
edition.workspace = true
7+
license.workspace = true
8+
readme.workspace = true
9+
repository.workspace = true
10+
rust-version.workspace = true
11+
version.workspace = true
12+
13+
[dependencies]
14+
rustix = { version = "1.0.0", default-features = false, features = ["fs", "std"] }
15+
16+
[dev-dependencies]
17+
proptest = "1"
18+
tempfile = { version = "3.8.0", default-features = false }
19+
20+
[lints]
21+
workspace = true

0 commit comments

Comments
 (0)