You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc
+57-4Lines changed: 57 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -46,7 +46,7 @@ If you want to use partitioning, you must specify a custom partition specificati
46
46
47
47
=== Manual deletion of Iceberg tables
48
48
49
-
The AWS Glue catalog integration requires Redpanda Iceberg tables to be manually deleted. To manually delete Iceberg tables, you must first set the cluster property config_ref:iceberg_delete,true,properties/cluster-properties[`iceberg_delete`] to `false` when you configure the catalog integration.
49
+
The AWS Glue catalog integration does not support automatic deletion of Iceberg tables from Redpanda. To manually delete Iceberg tables in AWS Glue, you must first set the cluster property config_ref:iceberg_delete,true,properties/cluster-properties[`iceberg_delete`] to `false` when you configure the catalog integration.
50
50
51
51
When `iceberg_delete` is set to `false`, you can delete the Redpanda topic, and then delete the table in AWS Glue and the Iceberg data and metadata files in the S3 bucket. If your intent is to recreate the topic after deleting it, you are required to delete the table data entirely before recreating the topic.
52
52
@@ -56,11 +56,36 @@ ifndef::env-cloud[]
56
56
You must allow Redpanda access to AWS Glue services in your AWS account. You can use the same access credentials that you configured for S3 (IAM role, access keys, and KMS key), as long as you have also added read and write access to AWS Glue Data Catalog.
57
57
58
58
For example, you could create a separate IAM policy that manages access to AWS Glue, and attach it to the IAM role that Redpanda also uses to access S3. It is recommended to add all AWS Glue API actions in the policy (`"glue:*"`) on the following resources:
59
+
60
+
- Root catalog (`catalog`)
61
+
- All databases (`database/*`)
62
+
- All tables (`table/\*/*`)
63
+
64
+
Your IAM policy should include a statement similar to the following:
You must allow Redpanda access to AWS Glue services in your AWS account. It is recommended to create a new IAM policy or role that manages access to AWS Glue, allowing all AWS Glue API actions (`"glue:*"`) on the following resources:
63
-
endif::[]
64
89
65
90
- Root catalog (`catalog`)
66
91
- All databases (`database/*`)
@@ -83,10 +108,38 @@ Your IAM policy should include a statement similar to the following:
For more information on configuring IAM permissions, see the https://docs.aws.amazon.com/glue/latest/dg/configure-iam-for-glue.html[AWS Glue documentation^].
92
145
@@ -125,13 +178,13 @@ Run `rpk cluster config edit` to update these properties:
125
178
----
126
179
iceberg_enabled: true
127
180
# Glue requires Redpanda Iceberg tables to be manually deleted
Copy file name to clipboardExpand all lines: modules/reference/pages/properties/cluster-properties.adoc
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2265,7 +2265,7 @@ Source of AWS credentials for Iceberg REST catalog SigV4 authentication. If not
2265
2265
endif::[]
2266
2266
2267
2267
ifdef::env-cloud[]
2268
-
Source of AWS credentials for Iceberg REST catalog SigV4 authentication. If using `iceberg_rest_catalog_aws_access_key` and `iceberg_rest_catalog_aws_secret_key` for Glue catalog authentication, you must set this property to `config_file`.
2268
+
Source of AWS credentials for Iceberg REST catalog SigV4 authentication. If providing explicit credentials using `iceberg_rest_catalog_aws_access_key` and `iceberg_rest_catalog_aws_secret_key` for Glue catalog authentication, you must set this property to `config_file`.
0 commit comments