Discuss feature naming for optional transport and crypto support

[kevinjqliu]

Context

PR #724 makes reqwest optional by introducing base provider features such as:

• cloud-base
• aws-base
• azure-base
• gcp-base
• http-base

PR #707 makes the crypto provider configurable by separating the provider implementation from the default bundled crypto choice.

Together, these changes introduce two independent configuration axes:

1. HTTP transport

   • use the built-in reqwest transport
   • or provide a custom HTTP connector

2. Crypto provider

   • use the default bundled crypto provider
   • or provide/configure crypto externally

This raises a naming question for the feature model before the next release.

Proposal

Use *-base for provider implementation features, and compose transport and crypto independently.

cloud-base = [
  # shared cloud implementation deps, no default reqwest or bundled crypto choice
]

reqwest = [
  "dep:reqwest",
  "reqwest/stream",
]

aws-lc-rs = [
  "dep:aws-lc-rs",
]

ring = [
  "dep:ring",
]

cloud = [
  "cloud-base",
  "reqwest",
  "aws-lc-rs",
]

aws-base = [
  "cloud-base",
  "crc-fast",
  "md-5",
]

aws = [
  "aws-base",
  "reqwest",
  "aws-lc-rs",
]

azure-base = [
  "cloud-base",
  "httparse",
]

azure = [
  "azure-base",
  "reqwest",
  "aws-lc-rs",
]

gcp-base = [
  "cloud-base",
  "rustls-pki-types",
]

gcp = [
  "gcp-base",
  "reqwest",
  "aws-lc-rs",
]

http-base = [
  "cloud-base",
]

http = [
  "http-base",
  "reqwest",
  "aws-lc-rs",
]

Rationale

• cloud-base means shared cloud implementation without the default transport or bundled crypto choice.
• *-base means provider-specific implementation without the default transport or bundled crypto choice.
• reqwest means enabling the built-in HTTP transport.
• aws-lc-rs and ring represent explicit crypto provider choices.
• aws, azure, gcp, and http preserve the existing batteries-included behavior.
• cloud is preserved for backwards compatibility.
• Provider features compose directly from *-base + reqwest + crypto-provider for clarity.

The core model is:

cloud-base = shared cloud implementation without default reqwest or bundled crypto

aws-base   = cloud-base + AWS/S3-specific implementation deps
azure-base = cloud-base + Azure-specific implementation deps
gcp-base   = cloud-base + GCS-specific implementation deps
http-base  = cloud-base + HTTP-specific implementation deps

reqwest    = built-in reqwest HTTP transport

aws-lc-rs  = bundled aws-lc-rs crypto provider
ring       = bundled ring crypto provider

cloud      = cloud-base + reqwest + default crypto provider
             compatibility alias for the old cloud behavior

aws        = aws-base + reqwest + default crypto provider
azure      = azure-base + reqwest + default crypto provider
gcp        = gcp-base + reqwest + default crypto provider
http       = http-base + reqwest + default crypto provider

Example advanced compositions

# S3 implementation only; user provides HTTP connector and crypto provider
object_store = { default-features = false, features = ["aws-base"] }

# S3 implementation + built-in reqwest, but user controls crypto provider
object_store = { default-features = false, features = ["aws-base", "reqwest"] }

# S3 implementation + custom HTTP connector, but object_store provides default crypto
object_store = { default-features = false, features = ["aws-base", "aws-lc-rs"] }

# S3 implementation + built-in reqwest + explicit crypto provider
object_store = { default-features = false, features = ["aws-base", "reqwest", "ring"] }

Open question

Should we standardize on this naming convention before the next release, so optional transport and optional crypto use the same composable feature model?

[tustvold]

This makes sense to me, only question is if people prefer core or base but I don't feel strongly

[kevinjqliu]

naming is hard 🥲

my preferences is:
-base > -core > -custom

but no strong feelings

[alamb]

base is fine with me too. Thanks @kevinjqliu

[alamb]

So, how shall we proceed? I feel like we need to merge

1. Make reqwest optional #724
2. Pluggable Crypto / Update reqwest 0.13 #707

And then document the various feature flags.

[kevinjqliu]

One idea from reviewing #724: since this is already a breaking-change release, should we remove cloud?

Once we introduce *-base, reqwest, and explicit crypto provider features, cloud becomes less clear. #724 keeps cloud for backwards compatibility:

# Compatibility/convenience alias.
# Historically, cloud meant "common cloud support", including reqwest.
# Keep that behavior for existing users.
cloud = ["cloud-base", "reqwest"]

Given that cloud is just cloud-base + reqwest, it does not seem useful to keep in a breaking-change release. Users who want the default setup can enable aws, azure, gcp, or http directly. Users who want custom transport or crypto can use the corresponding *-base feature and compose the pieces they need.

I would still keep cloud-base, though. It is still useful as the shared implementation feature that provider-specific base features compose from:

[kylebarron]

I'm +1 on removing cloud. I feel like cloud was always an "internal" feature, and end users should be pointed to aws, azure, gcp, and http