Apache Iceberg Rust version
0.6.0 (latest version)
Describe the bug
The spec requires writers to use a column's write-default when the writer does not supply a value for the column. IcebergTableProvider does not implement DataFusion's TableProvider::get_column_default, so DataFusion's insert planner falls back to NULL for columns omitted from an INSERT. Rows written through DataFusion into a table whose schema declares write-default values silently get NULL instead of the default.
For comparison, iceberg-java delegates write-default application to the engine in the same way: its Spark integration exposes write-default as the engine-level column default (TypeToSparkType), and Spark materializes the value during INSERT planning. The DataFusion integration is the corresponding place in iceberg-rust.
To Reproduce
-- table schema: id int (required), category string with write-default 'general'
INSERT INTO t (id) VALUES (1);
SELECT category FROM t; -- returns NULL, expected 'general'
Expected behavior
IcebergTableProvider implements get_column_default, mapping each top-level column's write-default literal to a DataFusion expression, so the insert planner fills omitted columns with the declared default.
Part of #2411 (Default Values).
Willingness to contribute
I can contribute a fix for this bug independently
Apache Iceberg Rust version
0.6.0 (latest version)
Describe the bug
The spec requires writers to use a column's
write-defaultwhen the writer does not supply a value for the column.IcebergTableProviderdoes not implement DataFusion'sTableProvider::get_column_default, so DataFusion's insert planner falls back toNULLfor columns omitted from anINSERT. Rows written through DataFusion into a table whose schema declareswrite-defaultvalues silently getNULLinstead of the default.For comparison, iceberg-java delegates write-default application to the engine in the same way: its Spark integration exposes
write-defaultas the engine-level column default (TypeToSparkType), and Spark materializes the value during INSERT planning. The DataFusion integration is the corresponding place in iceberg-rust.To Reproduce
Expected behavior
IcebergTableProviderimplementsget_column_default, mapping each top-level column'swrite-defaultliteral to a DataFusion expression, so the insert planner fills omitted columns with the declared default.Part of #2411 (Default Values).
Willingness to contribute
I can contribute a fix for this bug independently