feat(rest): support AWS SigV4 request signing for the REST catalog

[plusplusjiajia]

What

Adds AWS SigV4 request signing to the REST catalog client, for catalogs that authenticate REST requests with SigV4.

Approach

• SigV4Signer + a small HttpRequestSigner trait; the client signs each request after token + header.* extras. Follows Iceberg Java's RESTSigV4AuthSession (signs all headers minus a blacklist).
• PayloadHashMode handles the Iceberg detail: x-amz-content-sha256 is base64 for non-empty bodies (hex for empty); StandardAws keeps hex everywhere.
• Config: rest.sigv4-enabled, rest.signing-region/-name/-access-key-id/-secret-access-key/-session-token (falls back to AWS_* env), mutually

[dannycjones]

There's a PR open for SigV4 signing, is this picking up from that one? #2311

I ask as its had a few rounds of feedback already.

[plusplusjiajia]

There's a PR open for SigV4 signing, is this picking up from that one? #2311

I ask as its had a few rounds of feedback already.

@dannycjones Thanks for the pointer — I'd missed #2311, just took a look and compared the two. Mine isn't based on it: it follows Iceberg Java's RESTSigV4AuthSessionRESTSigV4AuthSession(apache/iceberg#11995) and the merged iceberg-cpp version(apache/iceberg-cpp#616). The main difference I see is the base64-encoded x-amz-content-sha256 convention (the Java behavior), which #2311's hex-only signing doesn't cover and which some REST servers require.
I'm keen to help get SigV4 support landed either way. Since #2311 is further along, I'm happy to fold the base64 support into it rather than duplicate — or carry this one forward if you'd prefer. Whatever helps move it forward.

[CTTY]

I haven't thought quite clearly about this part yet, my general intuition is that it would be better to start from something like a AuthManager so we have a clean interface before diving into the specific implementation.

Would be happy to hear more thoughts on this

[CTTY]

I think we need a more general design rather than just a signer. some authentication mechanism is token-based.

[plusplusjiajia]

@CTTY I dug into how Java structures this, and I think your point is well taken: OAuth2 token handling is hardcoded inside HttpClient today, and this PR adds a second, parallel mechanism that is mutually exclusive with token auth. In Java the two compose — SigV4AuthManager wraps a delegate session, relocates its Authorization header to X-Iceberg-Authorization, then signs — so SigV4-over-OAuth2 is a real combination the current design can't express.

Is something along these lines what you had in mind, mirroring Java's AuthManager/AuthSession?

#[async_trait]
pub trait AuthManager: Debug + Send + Sync {
    /// Session used for catalog-level requests.
    async fn catalog_session(
        &self,
        props: &HashMap<String, String>,
    ) -> Result<Arc<dyn AuthSession>>;
    // room to grow, matching Java: init_session() for the config
    // handshake, table_session() for table-scoped auth, close().
}

#[async_trait]
pub trait AuthSession: Debug + Send + Sync {
    /// Applies authentication to an outgoing request (headers, signing, ...).
    async fn authenticate(&self, request: &mut reqwest::Request) -> Result<()>;
}

with NoopAuthManager / OAuth2Manager (existing logic extracted, behavior unchanged) / SigV4AuthManager (wrapping a delegate, Java-style) as the initial implementations, selected via rest.auth.type or injected through the builder.

If that matches your intuition, my instinct would be to land the interface plus the OAuth2 extraction as a small standalone refactor first, then rework this PR on top as the SigV4 implementation — which would also give #2311 a common landing spot.

[plusplusjiajia]

Prototype is up as a draft PR: #2815 — full AuthManager/AuthSession shape with Noop/OAuth2/SigV4 managers; details and known simplifications in the PR description.

[CTTY]

I didn't know this detail until this PR. Thanks for capturing this!