Skip to content

Commit 3b6d052

Browse files
committed
Name change
1 parent afd13e4 commit 3b6d052

3 files changed

Lines changed: 14 additions & 13 deletions

File tree

modules/manage/partials/iceberg/about-iceberg-topics.adoc

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ For details on the partitioning specification, including allowed transforms, see
133133
--
134134
* `key_value`: Creates an Iceberg table using a simple schema, consisting of two columns, one for the record metadata including the key, and another binary column for the record's value.
135135
* `value_schema_id_prefix`: Creates an Iceberg table whose structure matches the Redpanda schema for this topic, with columns corresponding to each field. You must register a schema in the Schema Registry (see next step), and producers must write to the topic using the Schema Registry wire format. Redpanda parses the schema used by the record based on the schema ID encoded in the payload header and stores the topic values in the corresponding table columns.
136-
* `value_subject_latest`: Creates an Iceberg table whose structure matches the latest schema registered for the subject in the Schema Registry.
136+
* `value_schema_latest`: Creates an Iceberg table whose structure matches the latest schema registered for the subject in the Schema Registry.
137137
* `disabled` (default): Disables writing to an Iceberg table for this topic.
138138
--
139139
+
@@ -177,12 +177,13 @@ The Iceberg table resides in a namespace called `redpanda` and has the same name
177177

178178
== About schema support and translation to Iceberg format
179179

180-
The `redpanda.iceberg.mode` topic property determines how Redpanda maps the topic data to the Iceberg table structure. You can use the `value_schema_id_prefix` and `value_subject_latest` modes to generate an Iceberg table that matches the structure of an Avro or Protobuf schema in the Schema Registry, or you can use the `key_value` mode where Redpanda stores the record values as-is in the table.
180+
The `redpanda.iceberg.mode` topic property determines how Redpanda maps the topic data to the Iceberg table structure. You can use the `value_schema_id_prefix` and `value_schema_latest` modes to generate an Iceberg table that matches the structure of an Avro or Protobuf schema in the Schema Registry, or you can use the `key_value` mode where Redpanda stores the record values as-is in the table.
181181

182182
The JSON Schema format is not supported. If your topic data is in JSON, use the `key_value` mode.
183183

184+
// TODO: Remove conditional once topic property page is available in Cloud
184185
ifndef::env-cloud[]
185-
See the xref:reference:properties/topic-properties.adoc#redpanda-iceberg-mode[`redpanda.iceberg.mode` property reference] for more details, including allowed syntax for `value_subject_latest`.
186+
See the xref:reference:properties/topic-properties.adoc#redpanda-iceberg-mode[`redpanda.iceberg.mode` property reference] for more details, including allowed syntax for `value_schema_latest`.
186187
endif::[]
187188

188189
=== Iceberg modes and table schemas
@@ -231,7 +232,7 @@ CREATE TABLE ClickEvent (
231232

232233
Consider this approach if the topic data is in JSON, or if you can use the Iceberg data in its semi-structured format.
233234

234-
The `value_schema_id_prefix` and `value_subject_latest` modes translate to the following table format:
235+
The `value_schema_id_prefix` and `value_schema_latest` modes translate to the following table format:
235236

236237
[,sql]
237238
----
@@ -257,7 +258,7 @@ ifdef::env-cloud[]
257258
With schema integration using the `value_schema_id_prefix` mode, Redpanda uses the Schema Registry wire format, which embeds the magic byte and schema ID in the message payload header. Producers to the topic must use the wire format in the serialization process so Redpanda can determine the schema used for each record, parse the record according to that schema, and use the schema to define the Iceberg table. See also: the https://www.redpanda.com/blog/schema-registry-kafka-streaming#how-does-serialization-work-with-schema-registry-in-kafka[Understanding Apache Kafka Schema Registry^] blog post.
258259
endif::[]
259260

260-
The `value_subject_latest` mode uses the latest schema registered for the subject in the Schema Registry. This mode is similar to the `value_schema_id_prefix` mode, but it does not require that producers use the wire format.
261+
The `value_schema_latest` mode uses the latest schema registered for the subject in the Schema Registry. This mode is similar to the `value_schema_id_prefix` mode, but it does not require that producers use the wire format.
261262

262263
If Redpanda fails to translate the record to the columnar format as defined by the schema, it writes the record to a dead-letter queue (DLQ) table. See <<manage-dead-letter-queue,Manage dead-letter queue>> for more information.
263264

modules/manage/partials/iceberg/query-iceberg-topics.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ endif::[]
4343
{"user_id": 2324, "event_type": "BUTTON_CLICK", "ts": "2024-11-25T20:23:59.380Z"}
4444
----
4545

46-
=== Topic with schema (`value_schema_id_prefix` or `value_subject_latest` mode)
46+
=== Topic with schema (`value_schema_id_prefix` or `value_schema_latest` mode)
4747

4848
Assume that you have created the `ClickEvent` topic, set `redpanda.iceberg.mode` to `value_schema_id_prefix`, and are connecting to a REST-based Iceberg catalog. The following is an Avro schema for `ClickEvent`:
4949

modules/reference/pages/properties/topic-properties.adoc

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -538,24 +538,24 @@ Enable the Iceberg integration for the topic. You can choose one of three modes.
538538

539539
- `key_value`: Creates an Iceberg table with a `Key` column and a `Value` column. Redpanda stores the raw topic content in the `Value` column.
540540
- `value_schema_id_prefix`: Creates an Iceberg table whose structure matches the Redpanda schema for the topic, with columns corresponding to each field. Redpanda uses the Schema Registry wire format, consisting of the "magic byte" and schema ID encoded in the payload header, to parse the topic values per field and store them in the corresponding table columns.
541-
- `value_subject_latest`: Creates an Iceberg table whose structure matches the latest schema version in Schema Registry that matches the subject name. This mode is compatible with Avro and Protobuf schemas and is used when you don't produce to the topic using the wire format.
541+
- `value_schema_latest`: Creates an Iceberg table whose structure matches the latest schema version in Schema Registry that matches the subject name. This mode is compatible with Avro and Protobuf schemas and is used when you don't produce to the topic using the wire format.
542542
- `disabled`: Disables writing to an Iceberg table for the topic.
543543

544-
For `value_subject_latet` mode, use the following syntax:
544+
For `value_schema_latest` mode, use the following syntax:
545545

546546
[,bash]
547547
----
548-
value_subject_latest[:<key1>=<value1>,<key2>=<value2>]
548+
value_schema_latest[:<key1>=<value1>,<key2>=<value2>]
549549
----
550550

551-
* Only the string `value_subject_latest` is required in the property value.
551+
* Only the string `value_schema_latest` is required in the property value.
552552
* Key-value pairs as indicated by `<key>=<value>` inside the square brackets `[]` are optional. The key-value pairs override this mode's default behavior for the topic.
553553
** For Protobuf data, the mode's default behavior derives the suject name using xref:manage:schema-reg/schema-id-validation.adoc#set-subject-name-strategy-per-topic[TopicNameStrategy] and deserializes records using the first message definition in the associated Protobuf schema in the Schema Registry.
554554
+
555555
--
556-
* Specify a different subject name using a key-value pair `subject=<subject-name>`, for example `value_subject_latest:subject=my_protobuf_schema`.
557-
* Specify a different message definition using a key-value pair `protobuf_name=<message-name>`, for example `value_subject_latest:protobuf_name=com.example.manufacturing.SensorData`.
558-
* To specify both the subject and message definition, separate the key-value pairs with a comma, for example `value_subject_latest:protobuf_name=com.example.manufacturing.SensorData,subject=my_protobuf_schema`.
556+
* Specify a different subject name using a key-value pair `subject=<subject-name>`, for example `value_schema_latest:subject=my_protobuf_schema`.
557+
* Specify a different message definition using a key-value pair `protobuf_name=<message-name>`, for example `value_schema_latest:protobuf_name=com.example.manufacturing.SensorData`.
558+
* To specify both the subject and message definition, separate the key-value pairs with a comma, for example `value_schema_latest:protobuf_name=com.example.manufacturing.SensorData,subject=my_protobuf_schema`.
559559
--
560560
** For Avro data
561561

0 commit comments

Comments
 (0)