Skip to content

Commit fa53ce3

Browse files
dfa1claude
andcommitted
fix(writer): reject NUL bytes in field names
A NUL inside a column name aborts the reference toolchain: the Arrow FFI schema export hits a panic-cannot-unwind in arrow-rs (ffi_stream::get_schema) and SIGABRTs the whole process (measured against vortex-jni 0.75.0). Our pure-Java path handles NUL names fine, so without the guard we emit files the canonical reader dies on. Everything else is opaque UTF-8 and round-trips intact both directions — "", whitespace-only, "$$$$$", newlines, emoji — all measured and recorded in docs/compatibility.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent d3b5b25 commit fa53ce3

3 files changed

Lines changed: 28 additions & 0 deletions

File tree

docs/compatibility.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ resolves only the standalone decoders in `reader`; no encoder class is loaded.
3636
| `vortex.variant` arbitrary nested objects | Rust (`vortex.parquet.variant`) | ⚠️ Java encodes/decodes variant columns of **typed scalar** values (constant / chunked-of-constants core, optional shredded child); Java↔Rust round-trip verified. Arbitrary nested JSON objects and real path-based shredding need the `vortex.parquet.variant` physical encoding — deferred ([ADR 0014](../adr/0014-variant-encoding-strategy.md)). |
3737
| Arrow extension array import affecting Variant shape | Rust 0.74.0 (#8125) | Untested. Re-run integration fixtures against v0.74.0 once published. |
3838
| Duplicate struct field names | Rust writer rejects ("StructLayout must have unique field names"); Rust reader tolerates foreign files (first-match access) | ⚠️ Deliberate divergence on read: Java rejects such files with `VortexException("duplicate field name in file schema")` instead of tolerating them — the name-keyed `Chunk` API cannot represent both columns, and silent column loss is worse than a loud failure on a file the reference writer refuses to produce. Java's writer mirrors the Rust writer's rejection. Empty column name `""` is legal on both sides (pinned by `ColumnNameEdgeCasesIntegrationTest`). |
39+
| NUL (`U+0000`) in a field name | Aborts the Rust toolchain: Arrow FFI schema export hits a panic-cannot-unwind in `arrow-rs` (`ffi_stream::get_schema`) and SIGABRTs the process (measured against vortex-jni 0.75.0) | ⚠️ Java's writer rejects NUL in field names (`IllegalArgumentException`) so we never emit a file the canonical reader dies on; Java's reader tolerates such files (pure-Java path, no C strings). All other names are opaque UTF-8 and round-trip intact both directions — including `""`, whitespace-only, `$`-runs, newlines, and emoji (measured). |
3940

4041
## Encodings
4142

writer/src/main/java/io/github/dfa1/vortex/writer/VortexWriter.java

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,13 @@ private VortexWriter(
140140
if (!uniqueNames.add(name)) {
141141
throw new IllegalArgumentException("duplicate field name: " + name);
142142
}
143+
// A NUL byte in a field name aborts the reference toolchain's Arrow FFI export
144+
// (panic-cannot-unwind in arrow-rs ffi_stream::get_schema, SIGABRT — measured
145+
// 2026-07-04): never emit a file the canonical reader dies on. Everything else,
146+
// including "" and whitespace-only names, is legal and round-trips.
147+
if (name.indexOf('\u0000') >= 0) {
148+
throw new IllegalArgumentException("field name contains NUL (U+0000)");
149+
}
143150
}
144151
this.channel = channel;
145152
this.schema = schema;

writer/src/test/java/io/github/dfa1/vortex/writer/VortexWriterTest.java

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,26 @@ void create_duplicateFieldNames_throwsIllegalArgumentException(@TempDir Path tmp
9797
}
9898
}
9999

100+
@Test
101+
void create_fieldNameWithNulByte_throwsIllegalArgumentException(@TempDir Path tmp) throws IOException {
102+
// Given — a NUL byte inside a field name. Legal in our pure-Java stack, but it aborts
103+
// the reference toolchain's Arrow FFI export (panic-cannot-unwind in arrow-rs, SIGABRT,
104+
// measured 2026-07-04) — so the writer must never emit it. Whitespace-only and empty
105+
// names stay legal (see ColumnNameEdgeCasesIntegrationTest).
106+
var schema = new DType.Struct(
107+
List.of("col\u0000hidden"),
108+
List.of(new DType.Primitive(PType.I64, false)),
109+
false);
110+
Path file = tmp.resolve("nul.vtx");
111+
112+
// When / Then
113+
try (var ch = FileChannel.open(file, StandardOpenOption.CREATE, StandardOpenOption.WRITE)) {
114+
assertThatThrownBy(() -> VortexWriter.create(ch, schema, WriteOptions.defaults()))
115+
.isInstanceOf(IllegalArgumentException.class)
116+
.hasMessageContaining("NUL");
117+
}
118+
}
119+
100120
// ── writeChunk validation ─────────────────────────────────────────────────
101121

102122
@Test

0 commit comments

Comments
 (0)