Skip to content

Import cloud-dependencies-bom for Azure, AWS, and GCS version management - #355

Merged
aaime merged 2 commits into
geosolutions-it:masterfrom
groldan:centralize_cloud_libs
Feb 26, 2026
Merged

Import cloud-dependencies-bom for Azure, AWS, and GCS version management#355
aaime merged 2 commits into
geosolutions-it:masterfrom
groldan:centralize_cloud_libs

Conversation

@groldan

@groldan groldan commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

Replace per-module SDK version properties and Netty BOM workarounds with io.tileverse:cloud-dependencies-bom, which centrally manages cloud SDK versions, Netty exclusions, and dependency convergence.

Removes local azure.version and google-cloud-storage.version properties.
Adds azure-core-http-jdk-httpclient as the Netty-free HTTP transport.

See https://github.com/tileverse-io/cloud-dependencies-bom

From now on, we just need to make sure the same BOM version is imported here and in GeoWebCache. See


Original proposal

This is a proposal to have a centralized BOM to manage cloud storage dependencies (S3, GCS, Azure) dependencies across all projects (imageio-ext, geowebcache, geotools, geoserver, tileverse).

The approach addresses two dependency hell pain points:

  • Inconsistence cloud stoarge dependency versions across all projects, or rather the need to keep them in sync
  • Azure Blob Storage and AWS S3 SDKs depend on different versions of Netty

In both cases, Netty is the "default" HTTP Client implementation, and both use an SPI mechanism to load the client factory.

The tileverse-dependencies BOM removes Netty from both and provides official alternative HTTP-client implementations.

For Azure, a managed version of com.azure:azure-core-http-jdk-httpclient is provided.

For S3, software.amazon.awssdk:aws-crt-client. This one provides both Sync and Async HTTP clients, and claims to provide enhanced performance and reliability (see https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/crt-based-s3-client.html)

The AWS CRT-based S3 client improves transfer reliability in case there is a network failure.
Reliability is improved by retrying individual failed parts of a file transfer without restarting
the transfer from the beginning.

In addition, the AWS CRT-based S3 client offers enhanced connection pooling and
Domain Name System (DNS)load balancing, which also improves throughput.

You can use the AWS CRT-based S3 client in place of the SDK's standard S3 asynchronous
client and take advantage of its improved throughput right away.


This approach has been implemented already for the PMTiles GeoTools DataStore, and is used in production by GeoServer Cloud, and will be used in following modules for pure-java geoparquet DataStore, Versatiles containers (akin to PMTiles), Zarr, probably COPC and other cloud-native data formats.

Hence the proposal to implement it cross-projects and unify dependency management and convergence.


Further proposal:

The Tileverse project is an Apache2-liensed OS project, deployed to maven central, and open to collaboration.

For the sake of having a managed cloud storage libraries BOM, we could create a tileverse-cloudstorage-bom repository, versioned independently of the tileverse libraries, and use that one as the single source of truth for cloud storage dependency management across the involved projects, with shared governance (e.g. the GeoServer PSC).

Fixes the "IllegalState Could not find a valid Docker environment.
Please see logs and check configuration" error in CI builds after the
docker version upgrade in ubuntu-latest runners.
@groldan
groldan force-pushed the centralize_cloud_libs branch 3 times, most recently from 960f8c6 to 53e682e Compare February 20, 2026 13:24
@groldan groldan changed the title Use the tileverse-dependencies BOM for centralized, netty-free cloud libraries dep management Import cloud-dependencies-bom for Azure, AWS, and GCS version management Feb 20, 2026
@aaime

aaime commented Feb 23, 2026

Copy link
Copy Markdown
Member

I had a go at this and the GWC updates:

  • Confirmed it builds and passes online tests (did not try interactive testing)
  • Confirmed it reduces the netty dependencies, the only one that's left is in the google COG flavor, but it's a repackaged one, "io.grpc:grpc-netty-shaded:jar:1.76.2:runtime", not sure if it could be excluded too

Overall it looks good. Do you have an opportunity to also run some interactive testing?

@groldan

groldan commented Feb 24, 2026

Copy link
Copy Markdown
Contributor Author

hey, I'm working on a geoserver branch to update the assembly descriptors, then I'll report back on manual testing

Replace per-module SDK version properties, Netty BOM convergence profile,
and Jackson BOM workaround with io.tileverse:cloud-dependencies-bom,
which centrally manages cloud SDK versions, Netty exclusions, and
dependency convergence across the geospatial Java ecosystem.

Removes local aws.version, azure.version properties, the google-cloud
libraries-bom import, and the dependencyConvergence profile that manually
aligned Netty and Jackson versions. Adds aws-crt-client and
azure-core-http-jdk-httpclient as Netty-free HTTP transports.

See https://github.com/tileverse-io/cloud-dependencies-bom
@groldan
groldan force-pushed the centralize_cloud_libs branch from 3f15171 to 0a12de1 Compare February 25, 2026 18:19
@groldan
groldan marked this pull request as ready for review February 25, 2026 18:21
@groldan

groldan commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

@aaime @dromagnoli please check out #354 in order for the COG integration tests to run

@aaime
aaime merged commit 2603ca7 into geosolutions-it:master Feb 26, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants