Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions modules/manage/pages/iceberg/choose-iceberg-mode.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,16 @@ The latest schema is cached periodically. The cache period is defined by the clu

Default for `redpanda.iceberg.mode`. Disables writing to an Iceberg table for the topic.

[NOTE]
====
The following modes are compatible with producing to an Iceberg topic using Redpanda Console:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rockwotj mind signing off on this? Thank you

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this information is correct

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rockwotj did we confirm its impossible to get Console to product using the wire format (whereas rpk always does!) cc @r-vasquez

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Console only produces raw values, it does no encoding, so no support for protobuf/avro with or without schema registry.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be cool to support json or textproto for console testing (with or without wire format)


- `key_value`
- Starting in version 25.2, `value_schema_latest` with a JSON schema
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: The PR to add JSON Schema support is open and will be merged when 25.2 is GA.


Otherwise, records may fail to write to the Iceberg table and instead write to the xref:manage:iceberg/about-iceberg-topics.adoc#manage-dead-letter-queue[dead-letter queue].
====

== Configure Iceberg mode for a topic

You can set the Iceberg mode for a topic when you create the topic, or you can update the mode for an existing topic.
Expand Down
4 changes: 1 addition & 3 deletions modules/manage/partials/iceberg/about-iceberg-topics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ TOPIC STATUS
<new-topic-name> OK
----

. Configure `redpanda.iceberg.mode` for the topic. You can choose one of the following modes:
. Configure `redpanda.iceberg.mode` for the topic. You can choose one of the following xref:manage:iceberg/choose-iceberg-mode.adoc[Iceberg modes]:
+
--
* `key_value`: Creates an Iceberg table using a simple schema, consisting of two columns, one for the record metadata including the key, and another binary column for the record's value.
Expand All @@ -154,8 +154,6 @@ rpk topic alter-config <new-topic-name> --set redpanda.iceberg.mode=<topic-icebe
TOPIC STATUS
<new-topic-name> OK
----
+
See also: xref:manage:iceberg/choose-iceberg-mode.adoc[].

. Register a schema for the topic. This step is required for the `value_schema_id_prefix` and `value_schema_latest` modes.
+
Expand Down
17 changes: 10 additions & 7 deletions modules/manage/partials/iceberg/use-iceberg-catalogs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ The Iceberg integration for Redpanda Cloud supports multiple Iceberg catalogs ac
The following matrix shows the current status of Iceberg integrations across different cloud providers and catalogs. Check this matrix regularly as Redpanda Cloud continues to expand GA coverage for Iceberg topics.

|===
| |Databricks Unity Catalog |Snowflake Open Catalog |Dremio Nessie
| | Databricks Unity Catalog | Snowflake Open Catalog | AWS Glue Data Catalog | Google BigQuery

|AWS |Beta |Beta |Beta
|Azure |Beta |Beta |Beta
|GCP |Beta |Beta |Beta
|AWS |Supported |Beta |Beta |N/A
|GCP |Supported |Beta |N/A |Beta
|Azure |Beta |Beta |N/A |N/A
|===
endif::[]

Expand All @@ -46,13 +46,16 @@ The following shows the current status of Iceberg catalog integrations. Check th
|===
|Catalog service | Status

|Databricks Unity Catalog |Tested
|Snowflake Open Catalog |Tested
|Dremio Nessie |Tested
|Databricks Unity Catalog |Supported
|Snowflake Open Catalog |Supported
|AWS Glue Data Catalog | Beta
|Google BigQuery |Beta

|===
endif::[]

Other REST catalogs such as Dremio Nessie, Apache Polaris, and the Apache reference implementation have been tested but are not regularly verified. For more information, contact https://support.redpanda.com/hc/en-us/requests/new[Redpanda Support^].
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wzzzrd86 mind signing off on this? Thank you


=== Set cluster properties

To connect to a REST catalog, set the following cluster configuration properties:
Expand Down
20 changes: 20 additions & 0 deletions modules/reference/pages/properties/cluster-properties.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2090,6 +2090,26 @@ endif::[]

// end::iceberg_rest_catalog_authentication_mode[]

// tag::iceberg_rest_catalog_base_location[]

=== iceberg_rest_catalog_base_location

NOTE: This property is available in Redpanda version 25.1.7 and later.

Base URI for the Iceberg REST catalog. If unset, the REST catalog server determines the location. Some REST catalogs, like AWS Glue, require the client to set this. After Iceberg is enabled, do not change this value.

*Requires restart:* Yes

*Visibility:* `user`

*Type:* string

*Default:* `null`
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wdberkeley @alenkacz does this default also apply to Cloud?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes


---

// end::iceberg_rest_catalog_base_location[]

// tag::iceberg_rest_catalog_client_id[]
=== iceberg_rest_catalog_client_id

Expand Down