Skip to content

Commit cfa83dc

Browse files
committed
Clarify base location requirements
1 parent fb4fefd commit cfa83dc

1 file changed

Lines changed: 23 additions & 9 deletions

File tree

modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,12 @@ endif::[]
6767

6868
When `iceberg_delete` or the topic override `redpanda.iceberg.delete` is set to `false`, you can delete the Redpanda topic, and then delete the table in AWS Glue and the Iceberg data and metadata files in the S3 bucket. If you plan to re-create the topic after deleting it, you must delete the table data entirely before re-creating the topic.
6969

70+
ifdef::env-cloud[]
71+
=== Same AWS account required
72+
73+
For BYOC and BYOVPC deployments, the AWS Glue Data Catalog must be in the same AWS account as the cluster. Cross-account Glue access has not been tested or certified. If you need to use a Glue catalog in a different AWS account, contact https://support.redpanda.com[Redpanda support^].
74+
endif::[]
75+
7076
== Authorize access to AWS Glue
7177

7278
ifdef::env-cloud[]
@@ -172,8 +178,12 @@ iceberg_rest_catalog_base_location: s3://<bucket-name>/<warehouse-path>
172178
Use your own values for the following placeholders:
173179
+
174180
--
175-
- `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
176-
- `<bucket-name>` and `<warehouse-path>`: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3://<bucket-name>/iceberg`. As a security best practice, Redpanda Data recommends specifying a subfolder (using prefixes) rather than the root of the bucket.
181+
* `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
182+
* `<bucket-name>` and `<warehouse-path>`: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3://<bucket-name>/iceberg`. This must be the same bucket used for Tiered Storage (your `cloud_storage_bucket`). You cannot specify a different bucket for Iceberg data.
183+
+
184+
`<warehouse-path>` is a name you choose (such as `iceberg`) as the logical name for the warehouse represented by all Redpanda Iceberg topic data in the cluster.
185+
+
186+
As a security best practice, do not use the bucket root for the base location. Always specify a subfolder to avoid interfering with Tiered Storage data.
177187
--
178188
endif::[]
179189
ifdef::env-cloud[]
@@ -200,7 +210,7 @@ rpk cluster config set \
200210
iceberg_rest_catalog_authentication_mode=aws_sigv4 \
201211
iceberg_rest_catalog_credentials_source=aws_instance_metadata \
202212
iceberg_rest_catalog_aws_region=<glue-region> \
203-
iceberg_rest_catalog_base_location=s3://<bucket-name>/<warehouse-path>
213+
iceberg_rest_catalog_base_location=s3://<cluster-storage-bucket-name>/<warehouse-path>
204214
----
205215
--
206216
@@ -221,20 +231,24 @@ rpk cluster config set \
221231
iceberg_rest_catalog_aws_region=<glue-region> \
222232
iceberg_rest_catalog_aws_access_key=<glue-access-key> \
223233
iceberg_rest_catalog_aws_secret_key='${secrets.<glue-secret-key-name>}' \
224-
iceberg_rest_catalog_base_location=s3://<bucket-name>/<warehouse-path>
234+
iceberg_rest_catalog_base_location=s3://<cluster-storage-bucket-name>/<warehouse-path>
225235
----
226236
--
227237
======
228238
+
229239
Use your own values for the following placeholders:
230240
+
231241
--
232-
- `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in your config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
233-
- `<bucket-name>` and `<warehouse-path>`: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3://<bucket-name>/iceberg`. For BYOC clusters, the bucket name is `redpanda-cloud-storage-<cluster-id>`. For BYOVPC clusters, use the name of the bucket you created as a customer-managed resource.
242+
* `<glue-region>`: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in your config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property.
243+
* `<cluster-storage-bucket-name>` and `<warehouse-path>`: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3://<cluster-storage-bucket-name>/iceberg`.
244+
** Bucket name: For BYOC clusters, the bucket name is `redpanda-cloud-storage-<cluster-id>`. For BYOVPC clusters, use the name of the bucket you created as a customer-managed resource.
245+
+
246+
This must be the same bucket used for Tiered Storage. You cannot specify a different bucket for Iceberg data.
247+
** Warehouse: This is a name you choose as the logical name (such as `iceberg`) for the warehouse represented by all Redpanda Iceberg topic data in the cluster.
234248
+
235-
As a security best practice, Redpanda Data recommends specifying a subfolder (using prefixes) rather than the root of the bucket.
236-
- `<glue-access-key>` (static credentials only): The AWS access key ID for your Glue service account.
237-
- `<glue-secret-key-name>` (static credentials only): The name of the secret that stores the AWS secret access key for your Glue service account. To reference a secret in a cluster property, for example `iceberg_rest_catalog_aws_secret_key`, you must first xref:manage:iceberg/use-iceberg-catalogs.adoc#store-a-secret-for-rest-catalog-authentication[store the secret value].
249+
As a security best practice, do not use the bucket root for the base location. Always specify a subfolder to avoid interfering with Tiered Storage data.
250+
* `<glue-access-key>` (static credentials only): The AWS access key ID for your Glue service account.
251+
* `<glue-secret-key-name>` (static credentials only): The name of the secret that stores the AWS secret access key for your Glue service account. To reference a secret in a cluster property, for example `iceberg_rest_catalog_aws_secret_key`, you must first xref:manage:iceberg/use-iceberg-catalogs.adoc#store-a-secret-for-rest-catalog-authentication[store the secret value].
238252
--
239253
endif::[]
240254
+

0 commit comments

Comments
 (0)