Skip to content

No per-table credential vending in REST catalog #10

Description

@smaheshwar-pltr

Problem

iceberg-cpp's REST catalog has no support for per-table credential vending. When a REST catalog server returns vended credentials (temporary S3 access keys) in the LoadTableResponse, iceberg-cpp ignores them and uses the catalog-level FileIO for all tables. This means consumers built on iceberg-cpp cannot access tables in environments where the REST catalog vends per-table scoped S3 credentials.

Consumers using iceberg-cpp to read tables from a credential-vending REST catalog get authentication failures when trying to read data files, because no S3 credentials are propagated from the LoadTableResponse to the table's FileIO.

Error

When scanning a table whose data files require vended credentials:

HTTP 403 Forbidden
AccessDenied: Access Denied
Authentication Failure - this is usually caused by invalid or missing credentials.
* No credentials are provided.

The catalog discovery and table metadata loading succeed (those use the catalog-level credentials), but reading the actual Parquet/Avro data files fails because the per-table S3 credentials from the REST catalog response are never extracted or applied.

Expected behavior

Per the Iceberg REST Catalog spec, when X-Iceberg-Access-Delegation: vended-credentials is sent, the server returns temporary S3 credentials in LoadTableResponse.config. These should be:

  1. Extracted from the response
  2. Merged with catalog properties (catalog < table config < storage credentials, with longest prefix match)
  3. Used to create a per-table FileIO
  4. Accessible to consumers via FileIO::properties() (matching the pattern in iceberg-java and PyIceberg)

Reference implementations

  • Java: RESTSessionCatalog sends X-Iceberg-Access-Delegation and resolves per-table FileIO. Trino reads credentials via baseTable.io().properties().
  • Python: PyIceberg's REST catalog merges vended credentials into table.io.properties for engine consumption.
  • Rust: Credentials flow through table.file_io().config().props().

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions