GH-563: Make ColumnMetaData.path_in_schema optional#564
GH-563: Make ColumnMetaData.path_in_schema optional#564etseidl wants to merge 3 commits intoapache:masterfrom
ColumnMetaData.path_in_schema optional#564Conversation
|
I hope to have a Java PoC available soon. |
|
Java PoC apache/parquet-java#3470 I've so far confirmed that parquet-cli cat from the Java PoC can read a file lacking |
|
I think it is a great idea -- though before merging this I think we should do a formal approval on the mailing list |
For sure! 👍 I just wanted to put up a concrete proposal to drive the discussion. Also, FWIW, I've started on an arrow-cpp PoC. We'll see how far I get 😅 |
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
|
C++ PoC apache/arrow#49707 |
| * the schema, and redundantly storing it here can lead to unnecessary | ||
| * bloat in the footer. Writers are encouraged to make the writing of | ||
| * this field optional, but for maximal compatibility should default to | ||
| * writing the field until at least Month 202X. |
There was a problem hiding this comment.
Based on "Forward incompatible features/changes should not be turned on by default until 2 years after the parquet-java implementation containing the feature is released." Lets maybe fill in the date as September 2028, assuming we get things merged by a september java release?
There was a problem hiding this comment.
I've gone ahead and put Sept 2028 in the text for now. We can update as needed later.
3 participants
Rationale for this change
path_in_schemaoptional #563.What changes are included in this PR?
Change
path_in_schemato optional.Do these changes have PoC implementations?
Yes.
path_in_schemaoptional arrow-rs#9678path_in_schemaoptional parquet-java#3470path_in_schemaoptional arrow#49707Closes #563