diff --git a/README.md b/README.md index da164a7b..80c49474 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,20 @@ It's possible to build `object_store` for the `wasm32-unknown-unknown` target, h cargo build -p object_store --target wasm32-unknown-unknown ``` +## Disabling `reqwest` + +The `aws`, `azure`, `gcp`, and `http` features each bundle a [`reqwest`]-based HTTP transport. To target [`wasm32-wasip1`] (where `reqwest` does not compile) or otherwise supply your own HTTP client, build against the matching `*-base` feature instead: + +``` +cargo build -p object_store --no-default-features --features aws-base --target wasm32-wasip1 +``` + +See [Disabling `reqwest`] in the crate docs for full details. + +[`reqwest`]: https://crates.io/crates/reqwest +[`wasm32-wasip1`]: https://doc.rust-lang.org/rustc/platform-support/wasm32-wasip1.html +[Disabling `reqwest`]: https://docs.rs/object_store/latest/object_store/#disabling-reqwest + ## Related Apache Crates Here are several related crates in different repositories from other Apache projects. diff --git a/src/aws/mod.rs b/src/aws/mod.rs index 42d84a43..c4d8063b 100644 --- a/src/aws/mod.rs +++ b/src/aws/mod.rs @@ -17,6 +17,9 @@ //! An object store implementation for S3 //! +//! See the [Feature Flags](crate#feature-flags) section for the difference +//! between the `aws` and `aws-base` features. +//! //! ## Multipart uploads //! //! Multipart uploads can be initiated with the [`ObjectStore::put_multipart_opts`] method. diff --git a/src/azure/mod.rs b/src/azure/mod.rs index 6c4968d6..11567433 100644 --- a/src/azure/mod.rs +++ b/src/azure/mod.rs @@ -17,6 +17,9 @@ //! An object store implementation for Azure blob storage //! +//! See the [Feature Flags](crate#feature-flags) section for the difference +//! between the `azure` and `azure-base` features. +//! //! ## Streaming uploads //! //! [`ObjectStore::put_multipart_opts`] will upload data in blocks and write a blob from those blocks. diff --git a/src/gcp/mod.rs b/src/gcp/mod.rs index 3402057c..7c663b86 100644 --- a/src/gcp/mod.rs +++ b/src/gcp/mod.rs @@ -17,6 +17,9 @@ //! An object store implementation for Google Cloud Storage //! +//! See the [Feature Flags](crate#feature-flags) section for the difference +//! between the `gcp` and `gcp-base` features. +//! //! ## Multipart uploads //! //! [Multipart uploads](https://cloud.google.com/storage/docs/multipart-uploads) diff --git a/src/http/mod.rs b/src/http/mod.rs index 3fbf162f..1c83d45b 100644 --- a/src/http/mod.rs +++ b/src/http/mod.rs @@ -19,6 +19,9 @@ //! //! This follows [rfc2518] commonly known as [WebDAV] //! +//! See the [Feature Flags](crate#feature-flags) section for the difference +//! between the `http` and `http-base` features. +//! //! Basic get support will work out of the box with most HTTP servers, //! even those that don't explicitly support [rfc2518] //! diff --git a/src/lib.rs b/src/lib.rs index 48711bb2..b2b199ee 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -96,6 +96,8 @@ doc = "* [`http`]: [HTTP/WebDAV Storage](https://datatracker.ietf.org/doc/html/rfc2518). See [`HttpBuilder`](http::HttpBuilder)" )] //! +//! See [Feature Flags](#feature-flags) for the full set of flags. +//! //! # Why not a Filesystem Interface? //! //! The [`ObjectStore`] interface is designed to mirror the APIs @@ -517,6 +519,51 @@ //! [Apache Iceberg]: https://iceberg.apache.org/ //! [Delta Lake]: https://delta.io/ //! +//! # Feature Flags +//! +//! The feature set is layered so that you can pick a provider independently +//! of its HTTP transport: +//! +//! * `cloud-base` holds the shared provider implementation (XML/JSON parsing, +//! credentials, retry, etc.) and intentionally does *not* depend on +//! `reqwest`. +//! * `reqwest` enables the built-in [`reqwest`]-based [`HttpConnector`]. +//! * `-base` (`aws-base`, `azure-base`, `gcp-base`, `http-base`) +//! adds the per-provider logic on top of `cloud-base` without pulling in +//! `reqwest`. +//! * `` (`aws`, `azure`, `gcp`, `http`) is the batteries-included +//! alias for `-base` + `reqwest` and is the typical choice. +//! +//! ## Provider features +//! +//! | Feature | Enables | Notes | +//! | --- | --- | --- | +//! | `aws` | `aws-base` + `reqwest` | Amazon S3 with the built-in HTTP transport. | +//! | `azure` | `azure-base` + `reqwest` | Azure Blob Storage with the built-in HTTP transport. | +//! | `gcp` | `gcp-base` + `reqwest` | Google Cloud Storage with the built-in HTTP transport. | +//! | `http` | `http-base` + `reqwest` | HTTP/WebDAV with the built-in HTTP transport. | +//! | `aws-base` | provider only | S3 provider without `reqwest`; supply your own [`HttpConnector`]. | +//! | `azure-base` | provider only | Azure provider without `reqwest`; supply your own [`HttpConnector`]. | +//! | `gcp-base` | provider only | GCS provider without `reqwest`; supply your own [`HttpConnector`]. | +//! | `http-base` | provider only | HTTP/WebDAV provider without `reqwest`; supply your own [`HttpConnector`]. | +//! +//! ## Transport and shared features +//! +//! | Feature | Description | +//! | --- | --- | +//! | `reqwest` | Enables the default [`reqwest`]-based [`HttpConnector`]. Pulled in automatically by `aws`, `azure`, `gcp`, and `http`. | +//! | `tls-webpki-roots` | When `reqwest` is enabled, also bundle Mozilla's [`webpki-roots`] CA certificates. See [TLS Certificates](#tls-certificates). | +//! | `cloud-base` | Shared cloud-provider implementation. Pulled in automatically by every `*-base` feature; usually not enabled directly. | +//! | `cloud` | Back-compat alias for `cloud-base` + `reqwest`. Kept for users that previously depended on the `cloud` umbrella feature. | +//! +//! ## Other features +//! +//! | Feature | Description | +//! | --- | --- | +//! | `fs` *(default)* | Local filesystem store via [`LocalFileSystem`](local::LocalFileSystem). | +//! | `tokio` | Enables Tokio-based utilities such as [`BufReader`](buffered::BufReader) and [`BufWriter`](buffered::BufWriter). Pulled in automatically by `fs` and the `*-base` features. | +//! | `integration` | Exposes the [`integration`] module, a reusable test suite for verifying custom [`ObjectStore`] implementations. Not API-stable. | +//! //! # TLS Certificates //! //! Stores that use HTTPS/TLS (this is true for most cloud stores) can choose the source of their [CA] @@ -533,9 +580,41 @@ //! Many [`ObjectStore`] implementations permit customization of the HTTP client via //! the [`HttpConnector`] trait and utilities in the [`client`] module. //! Examples include injecting custom HTTP headers or using an alternate -//! tokio Runtime I/O requests. +//! tokio Runtime for I/O requests. To replace `reqwest` entirely (rather than +//! tweak the bundled transport) see [Disabling `reqwest`](#disabling-reqwest). //! //! [`HttpConnector`]: client::HttpConnector +//! +//! # Disabling `reqwest` +//! +//! The `aws`, `azure`, `gcp`, and `http` features each bundle a +//! [`reqwest`]-based HTTP transport, which is the right choice for most +//! applications. If you would rather supply your own HTTP client — for example +//! to share an existing client, to target a platform where `reqwest` does not +//! compile (such as `wasm32-wasip1`), or to keep `reqwest` out of your +//! dependency tree — use the matching `*-base` feature and provide an +//! [`HttpConnector`](client::HttpConnector) at builder time. +//! +//! Remember to disable the default features so that `fs` (and its transitive +//! dependencies) is not pulled in: +//! +//! ```toml +//! [dependencies] +//! object_store = { version = "0.13", default-features = false, features = ["aws-base"] } +//! ``` +//! +//! ```ignore +//! use object_store::aws::AmazonS3Builder; +//! +//! let store = AmazonS3Builder::from_env() +//! // `my_connector` is your own `impl HttpConnector` +//! .with_http_connector(my_connector) +//! .build()?; +//! ``` +//! +//! See [Feature Flags](#feature-flags) above for the full set of flags. +//! +//! [`reqwest`]: https://crates.io/crates/reqwest #[cfg(feature = "aws-base")] pub mod aws;