Core: Read and write geometry and geography values in Avro

[huan233usc]

Summary

TypeToSchema mapped every Iceberg primitive to an Avro schema except geometry and geography,
so converting a schema with a geo column threw UnsupportedOperationException: Unsupported type ID: GEOMETRY before any value could be read or written through the Avro object model. This wires up the
value path.

Per the Avro type mapping in the spec, geometry and geography are stored as an Avro bytes field
carrying WKB — the same representation as binary:

Iceberg type	Avro type
binary	bytes
geometry	bytes (WKB)
geography	bytes (WKB)

So TypeToSchema.primitive() now maps both geo types to the existing BINARY_SCHEMA. The value
read/write paths dispatch on the Avro physical type, so once the schema is bytes the existing
byteBuffers reader/writer handle geo unchanged in both the generic and internal object models (geo
values are WKB ByteBuffers, exactly like binary) — no value-path code changes are needed.

This also closes the parquet-avro object model gap: ParquetAvroWriter / ParquetAvroValueReaders
build their Avro schema through AvroSchemaUtil.convert, which routed into the same TypeToSchema
that threw on geo, so geo was unreachable through that path too. Fixing TypeToSchema makes it work
transitively, with no parquet-avro code change. This is the follow-up szehon-ho asked about on #16982
(which added the Parquet WKB value path and deliberately left the Avro object model as a follow-up).

Note on the reverse direction

SchemaToType intentionally still maps Avro bytes → binary. Plain Avro bytes carries no marker to
distinguish geometry/geography from binary, and Iceberg resolves the real column type from the
expected/table schema, never by reverse-inferring from the file's Avro schema — the same way binary
and other bytes-backed types already behave.

Test plan

Object-model coverage via the shared DataTest geo cases and the Avro encoder/writer entry points:

• Enable supportsGeospatial() on the Avro object-model tests — TestGenericAvro, TestInternalAvro,
  the generic-data TestGenericData, and TestAvroEncoderUtil (so geo also round-trips through the
  AvroEncoderUtil.encode/decode entry point, keeping the two generic-model Avro tests in parity).
  This round-trips geometry and geography across multiple CRS and edge algorithms with randomly
  generated WKB values.
• Add the GEOMETRY/GEOGRAPHY → ByteBuffer handling the generic Avro path needs in RandomAvroData
  and AvroTestHelpers, mirroring the existing RandomInternalData / InternalTestHelpers handling.
• Explicit WKB round-trip tests:
  • TestAvroDataWriter.testGeospatialWkbRoundTrip — geometry/geography through the Avro DataWriter
    → PlannedDataReader path, including nulls.
  • TestParquet.testGeospatialWkbRoundTrip — through ParquetAvroWriter → ParquetAvroValueReaders,
    covering the parquet-avro object model, including nulls.
• ./gradlew :iceberg-core:test --tests org.apache.iceberg.avro.TestGenericAvro --tests org.apache.iceberg.avro.TestInternalAvro --tests org.apache.iceberg.avro.TestAvroEncoderUtil --tests org.apache.iceberg.avro.TestAvroDataWriter
• ./gradlew :iceberg-data:test --tests org.apache.iceberg.data.avro.TestGenericData
• ./gradlew :iceberg-parquet:test --tests org.apache.iceberg.parquet.TestParquet

[wombatu-kun]

TestAvroEncoderUtil extends DataTestBase and round-trips generic records through this same TypeToSchema + RandomAvroData path via AvroEncoderUtil.encode/decode, and on main it mirrors this test's type flags exactly (supportsUnknown, supportsTimestampNanos, supportsVariant). Enabling supportsGeospatial() there as well covers geo through the encoder entry point for free and keeps the two generic-model Avro tests in parity. szehon-ho raised the same coverage-parity point on the sibling Parquet PR #16982.

[huan233usc]

Done — enabled supportsGeospatial() on TestAvroEncoderUtil in 67619a7, so geo now round-trips through the AvroEncoderUtil.encode/decode entry point too, keeping the two generic-model Avro tests in parity (good call on the #16982 coverage-parity precedent).

I also added explicit WKB round-trip tests while here:

• TestAvroDataWriter.testGeospatialWkbRoundTrip — geometry/geography through the Avro DataWriter → PlannedDataReader path, including nulls.
• TestParquet.testGeospatialWkbRoundTrip — through ParquetAvroWriter → ParquetAvroValueReaders,

[wombatu-kun]

RandomUtil.wkbPoint(double, double) in api/src/test/java/org/apache/iceberg/util/RandomUtil.java already encodes a little-endian WKB point, and it is on core's test classpath (RandomAvroData imports it). szehon-ho raised this exact duplication on the sibling Parquet PR #16982, and the merged TestParquetDataWriter.wkbPoint delegates: return ByteBuffer.wrap(RandomUtil.wkbPoint(xCoord, yCoord));. Doing the same here keeps WKB encoding in one place and drops the new java.nio.ByteOrder import.

[huan233usc]

Done in bcbef59 — TestAvroDataWriter.wkbPoint now delegates to RandomUtil.wkbPoint(xCoord, yCoord) and the java.nio.ByteOrder import is dropped, matching the merged TestParquetDataWriter.wkbPoint. Thanks for the pointer to the #16982 precedent.

[wombatu-kun]

The test plan in the PR body still describes the previous commit: it lists only the supportsGeospatial() flips plus the RandomAvroData/AvroTestHelpers cases, and does not mention TestAvroEncoderUtil, TestAvroDataWriter.testGeospatialWkbRoundTrip, or this test - so a PR titled Core: silently adds parquet-module coverage. Worth saying in the description (it becomes the squash commit message) that this closes the parquet-avro object model gap (ParquetAvroWriter / ParquetAvroValueReaders), which was unreachable for geo because AvroSchemaUtil.convert threw on geo types - the follow-up szehon-ho asked about on #16982.

[huan233usc]

Updated the PR description: the test plan now lists TestAvroEncoderUtil, both testGeospatialWkbRoundTrip tests, and a Summary paragraph noting this closes the parquet-avro object model gap (ParquetAvroWriter/ParquetAvroValueReaders, unreachable for geo because AvroSchemaUtil.convert routed into the TypeToSchema that threw) — the #16982 follow-up. It'll carry into the squash message.

[wombatu-kun]

These bytes are not WKB. That is harmless in testGeospatialFooterMetricsSkipParquetBounds, which writes them through a Types.BinaryType schema, but this test writes them through the geometry/geography schema, so TypeToMessageType annotates the column with LogicalTypeAnnotation.geometryType(crs). Parquet then attaches a real GeospatialStatistics.Builder to that column (ColumnValueCollector), and its update(Binary) runs JTS WKBReader.read and catches ParseException with a "Failed to parse WKB geometry, omit it from stats" warning. So the test passes while writing a file whose geometry values are unparseable and whose geospatial statistics are empty. RandomUtil.wkbPoint(...) produces real WKB here too and makes the test name accurate.

[huan233usc]

Great catch — fixed in bcbef59. testGeospatialWkbRoundTrip now uses RandomUtil.wkbPoint(30, 10) / wkbPoint(-5, 40), so the values are real WKB that the geospatial statistics builder parses successfully instead of silently omitting. The bytes in testGeospatialFooterMetricsSkipParquetBounds are left as-is since that test writes through a BinaryType schema (no WKB parsing), as you noted.

[huan233usc]

Thanks for the careful review @wombatu-kun! Addressed in bcbef59:

• testGeospatialWkbRoundTrip now uses real WKB via RandomUtil.wkbPoint(...) so the geospatial statistics builder actually parses the values (the previous arbitrary bytes were silently dropped from stats).
• TestAvroDataWriter.wkbPoint delegates to RandomUtil.wkbPoint and drops the ByteOrder import, matching the merged TestParquetDataWriter.
• Updated the PR description/test plan to cover TestAvroEncoderUtil and both round-trip tests, and to note this closes the parquet-avro object model gap (the Parquet: Read and write geometry and geography WKB values #16982 follow-up).