Skip to content

Commit 1826ae2

Browse files
kbatuigasclaude
andauthored
Iceberg: use sts for Iceberg credentials source with Glue + Cloud (#1714)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 3bf2fa3 commit 1826ae2

1 file changed

Lines changed: 9 additions & 7 deletions

File tree

modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -120,10 +120,10 @@ You must configure credentials for the AWS Glue Data Catalog integration in eith
120120

121121
* Allow Redpanda to use the same `cloud_storage_*` credential properties configured for S3. This is the recommended approach.
122122
* If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication:
123-
** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`] overrides config_ref:cloud_storage_credentials_source,true,properties/cluster-properties[`cloud_storage_credentials_source`]
124-
** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] overrides config_ref:cloud_storage_access_key,true,properties/cluster-properties[`cloud_storage_access_key`]
125-
** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] overrides config_ref:cloud_storage_secret_key,true,properties/cluster-properties[`cloud_storage_secret_key`]
126-
** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] overrides config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`]
123+
** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`] overrides config_ref:cloud_storage_credentials_source,true,properties/object-storage-properties[`cloud_storage_credentials_source`]
124+
** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] overrides config_ref:cloud_storage_access_key,true,properties/object-storage-properties[`cloud_storage_access_key`]
125+
** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] overrides config_ref:cloud_storage_secret_key,true,properties/object-storage-properties[`cloud_storage_secret_key`]
126+
** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] overrides config_ref:cloud_storage_region,true,properties/object-storage-properties[`cloud_storage_region`]
127127
endif::[]
128128

129129
ifdef::env-cloud[]
@@ -135,7 +135,9 @@ For an example cluster configuration that uses the same IAM credentials for both
135135
* If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication:
136136
+
137137
--
138-
** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. To use the cluster's IAM role, set the property to `aws_instance_metadata`. To use static credentials, set to `config_file`.
138+
** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]
139+
*** Set the property to `sts` if you want to use the cluster's default IAM role.
140+
*** Set to `config_file` if you want to scope Glue access through your own IAM user and policy instead of the cluster's default IAM role, or if you want to use static credentials.
139141
** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] (static credentials only)
140142
** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] (static credentials only), added as a secret value (see the <<update-cluster-configuration,next section>> for details)
141143
** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`]
@@ -185,7 +187,7 @@ Use your own values for the following placeholders:
185187
+
186188
--
187189
* `<custom-namespace>`: A unique namespace for this cluster's Iceberg tables. Each Redpanda cluster that writes to the same Glue catalog must use a distinct namespace to avoid table name collisions. If omitted, the default namespace `redpanda` is used.
188-
* `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
190+
* `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/object-storage-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
189191
* `<bucket-name>` and `<warehouse-path>`: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3://<bucket-name>/iceberg`. This must be the same bucket used for object storage (your `cloud_storage_bucket`). You cannot specify a different bucket for Iceberg data.
190192
+
191193
`<warehouse-path>` is a name you choose (such as `iceberg`) as the logical name for the warehouse represented by all Redpanda Iceberg topic data in the cluster.
@@ -217,7 +219,7 @@ rpk cluster config set \
217219
iceberg_catalog_type=rest \
218220
iceberg_rest_catalog_endpoint=https://glue.<glue-region>.amazonaws.com/iceberg \
219221
iceberg_rest_catalog_authentication_mode=aws_sigv4 \
220-
iceberg_rest_catalog_credentials_source=aws_instance_metadata \
222+
iceberg_rest_catalog_credentials_source=sts \
221223
iceberg_rest_catalog_aws_region=<glue-region> \
222224
iceberg_rest_catalog_base_location=s3://<cluster-storage-bucket-name>/<warehouse-path>
223225
----

0 commit comments

Comments
 (0)