Skip to content

Commit ef5a384

Browse files
paulohtb6Feediver1
andcommitted
Add missing properties in reference (#1353)
Co-authored-by: Joyce Fee <102751339+Feediver1@users.noreply.github.com>
1 parent 91379fd commit ef5a384

4 files changed

Lines changed: 139 additions & 20 deletions

File tree

modules/manage/pages/schema-reg/schema-id-validation.adoc

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -87,8 +87,8 @@ When <<enable-schema-id-validation,schema ID validation is enabled>>, Redpanda u
8787

8888
To customize the subject name strategy per topic, set the following client topic properties:
8989

90-
* Set `redpanda.key.schema.id.validation` to `true` to enable key schema ID validation for the topic, and set `redpanda.key.subject.name.strategy` to the desired subject name strategy for keys of the topic (default: `TopicNameStrategy`).
91-
* Set `redpanda.value.schema.id.validation` to `true` to enable value schema ID validation for the topic, and set `redpanda.value.subject.name.strategy` to the desired subject name strategy for values of the topic (default: `TopicNameStrategy`).
90+
* Set xref:reference:properties/topic-properties.adoc#redpandakeyschemavalidation[`redpanda.key.schema.id.validation`] to `true` to enable key schema ID validation for the topic, and set xref:reference:properties/topic-properties.adoc#redpandakeysubjectnamestrategy[`redpanda.key.subject.name.strategy`] to the desired subject name strategy for keys of the topic (default: `TopicNameStrategy`).
91+
* Set xref:reference:properties/topic-properties.adoc#redpandavalueschemavalidation[`redpanda.value.schema.id.validation`] to `true` to enable value schema ID validation for the topic, and set xref:reference:properties/topic-properties.adoc#redpandavaluesubjectnamestrategy[`redpanda.value.subject.name.strategy`] to the desired subject name strategy for values of the topic (default: `TopicNameStrategy`).
9292

9393
[NOTE]
9494
====
@@ -97,17 +97,17 @@ The `redpanda.` properties have corresponding `confluent.` properties.
9797
|===
9898
| Redpanda property | Confluent property
9999
100-
| `redpanda.key.schema.id.validation`
101-
| `confluent.key.schema.validation`
100+
| xref:reference:properties/topic-properties.adoc#redpandakeyschemavalidation[`redpanda.key.schema.id.validation`]
101+
| xref:reference:properties/topic-properties.adoc#confluentkeyschemavalidation[`confluent.key.schema.validation`]
102102
103-
| `redpanda.key.subject.name.strategy`
104-
| `confluent.key.subject.name.strategy`
103+
| xref:reference:properties/topic-properties.adoc#redpandakeysubjectnamestrategy[`redpanda.key.subject.name.strategy`]
104+
| xref:reference:properties/topic-properties.adoc#confluentkeysubjectnamestrategy[`confluent.key.subject.name.strategy`]
105105
106-
| `redpanda.value.schema.id.validation`
107-
| `confluent.value.schema.validation`
106+
| xref:reference:properties/topic-properties.adoc#redpandavalueschemavalidation[`redpanda.value.schema.id.validation`]
107+
| xref:reference:properties/topic-properties.adoc#confluentvalueschemavalidation[`confluent.value.schema.validation`]
108108
109-
| `redpanda.value.subject.name.strategy`
110-
| `confluent.value.subject.name.strategy`
109+
| xref:reference:properties/topic-properties.adoc#redpandavaluesubjectnamestrategy[`redpanda.value.subject.name.strategy`]
110+
| xref:reference:properties/topic-properties.adoc#confluentvaluesubjectnamestrategy[`confluent.value.subject.name.strategy`]
111111
|===
112112
113113
The `redpanda.*` and `confluent.*` properties are compatible. Either or both can be set simultaneously.

modules/manage/partials/tiered-storage.adoc

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1186,7 +1186,6 @@ rpk cluster config set cloud_storage_enable_segment_uploads true
11861186
For some applications, where the newest data is more valuable than historical data, data accumulation can be worse than data loss. In such cases, where you cannot afford to lose the most recently-produced data by rejecting produce requests after producers have filled the local disks during the period of paused uploads, there is a less safe pause and resume mechanism. This mechanism prioritizes the ability to receive new data over retaining data that cannot be uploaded when disks are full:
11871187

11881188
- Set the xref:reference:properties/object-storage-properties.adoc#cloud_storage_enable_remote_allow_gaps[`cloud_storage_enable_remote_allow_gaps`] cluster configuration property to `true`. This allows for gaps in the logs of all Tiered Storage topics in the cluster.
1189-
- Set the `redpanda.remote.allow_gaps` configuration property to `true`. This allows gaps for one specific topic. This topic-level configuration option overrides the cluster-level default.
11901189

11911190
When you pause uploads and set one of these properties to `true`, there may be gaps in the range of offsets stored in object storage. You can seamlessly resume uploads by setting `*allow_gaps` to `true` at either the cluster or topic level. If set to `false`, disk space could be depleted and produce requests would be throttled.
11921191

@@ -1199,19 +1198,12 @@ rpk cluster config set cloud_storage_enable_segment_uploads false
11991198
# To avoid overflow when allowing gaps in the log.
12001199
# In this example, data that is not uploaded to cloud storage may be
12011200
# deleted if a disk fills before uploads are resumed.
1202-
1203-
rpk topic alter-config $topic-name --set redpanda.remote.allowgaps=true
1204-
# Uploads are paused and gaps are allowed. Local retention is allowed
1205-
# to delete data before it's uploaded, therefore some data loss is possible.
12061201
...
12071202

12081203
rpk cluster config set cloud_storage_enable_segment_uploads true
12091204
# Uploads are resumed but there could be gaps in the offsets.
12101205
# Wait until you see that the `redpanda_cloud_storage_paused_archivers`
12111206
# metric is equal to zero, indicating that uploads have resumed.
1212-
1213-
# Disable the gap allowance previously set for the topic.
1214-
rpk topic alter-config $topic-name --set redpanda.remote.allowgaps=false
12151207
```
12161208

12171209
== Caching

modules/reference/pages/properties/object-storage-properties.adoc

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -744,6 +744,7 @@ When set to `true`, Redpanda can re-upload data for compacted topics to object s
744744

745745
---
746746

747+
[[cloud_storage_enable_remote_allow_gaps]]
747748
=== cloud_storage_enable_remote_allow_gaps
748749

749750
Controls the eviction of locally stored log segments when Tiered Storage uploads are paused. Set to `false` to only evict data that has already been uploaded to object storage. If the retained data fills the local volume, Redpanda throttles producers. Set to `true` to allow the eviction of locally stored log segments, which may create gaps in offsets.
@@ -822,7 +823,7 @@ Enables adjacent segment merging. The segments are reuploaded if there is an opp
822823

823824
=== cloud_storage_enable_segment_uploads
824825

825-
Controls the upload of log segments to Tiered Storage. If set to `false`, this property temporarily pauses all log segment uploads from the Redpanda cluster. When the uploads are paused, the <<cloud_storage_enable_remote_allow_gaps, `cloud_storage_enable_remote_allow_gaps`>> cluster configuration and `redpanda.remote.allowgaps` topic properties control local retention behavior.
826+
Controls the upload of log segments to Tiered Storage. If set to `false`, this property temporarily pauses all log segment uploads from the Redpanda cluster. When the uploads are paused, the <<cloud_storage_enable_remote_allow_gaps, `cloud_storage_enable_remote_allow_gaps`>> cluster configuration and xref:properties/topic-properties.adoc#redpandaremoteallowgaps[`redpanda.remote.allowgaps`] topic properties control local retention behavior.
826827

827828
*Requires restart:* No
828829

modules/reference/pages/properties/topic-properties.adoc

Lines changed: 127 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -738,4 +738,130 @@ Controls how often the data in the Iceberg table is refreshed with new data from
738738

739739
**Related topics**:
740740

741-
- xref:manage:iceberg/about-iceberg-topics.adoc[]
741+
- xref:manage:iceberg/about-iceberg-topics.adoc[]
742+
743+
---
744+
== Schema Registry and Validation Properties
745+
746+
The following properties control server-side schema ID validation for topics when using Schema Registry.
747+
748+
[[redpandakeyschemavalidation]]
749+
=== redpanda.key.schema.id.validation
750+
751+
Enable validation of the schema ID for keys on a record. When enabled, Redpanda validates that the schema ID encoded in the record's key is registered in the Schema Registry according to the configured subject name strategy.
752+
753+
*Type:* boolean
754+
755+
*Default:* `false`
756+
757+
**Related topics**:
758+
759+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
760+
761+
---
762+
[[redpandakeysubjectnamestrategy]]
763+
=== redpanda.key.subject.name.strategy
764+
765+
The subject name strategy for keys when `redpanda.key.schema.id.validation` is enabled. This determines how the topic and schema are mapped to a subject name in the Schema Registry.
766+
767+
*Type:* string
768+
769+
*Default:* `TopicNameStrategy`
770+
771+
*Accepted values:* [`TopicNameStrategy`, `RecordNameStrategy`, `TopicRecordNameStrategy`]
772+
773+
**Related topics**:
774+
775+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
776+
777+
---
778+
[[redpandavalueschemavalidation]]
779+
=== redpanda.value.schema.id.validation
780+
781+
Enable validation of the schema ID for values on a record. When enabled, Redpanda validates that the schema ID encoded in the record's value is registered in the Schema Registry according to the configured subject name strategy.
782+
783+
*Type:* boolean
784+
785+
*Default:* `false`
786+
787+
**Related topics**:
788+
789+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
790+
791+
---
792+
[[redpandavaluesubjectnamestrategy]]
793+
=== redpanda.value.subject.name.strategy
794+
795+
The subject name strategy for values when `redpanda.value.schema.id.validation` is enabled. This determines how the topic and schema are mapped to a subject name in the Schema Registry.
796+
797+
*Type:* string
798+
799+
*Default:* `TopicNameStrategy`
800+
801+
*Accepted values:* [`TopicNameStrategy`, `RecordNameStrategy`, `TopicRecordNameStrategy`]
802+
803+
**Related topic**:
804+
805+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
806+
807+
---
808+
[[confluentkeyschemavalidation]]
809+
=== confluent.key.schema.validation
810+
811+
Enable validation of the schema ID for keys on a record. This is a compatibility alias for `redpanda.key.schema.id.validation`. When enabled, Redpanda validates that the schema ID encoded in the record's key is registered in the Schema Registry according to the configured subject name strategy.
812+
813+
*Type:* boolean
814+
815+
*Default:* `false`
816+
817+
**Related topics*:
818+
819+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
820+
821+
---
822+
[[confluentkeysubjectnamestrategy]]
823+
=== confluent.key.subject.name.strategy
824+
825+
The subject name strategy for keys when `confluent.key.schema.validation` is enabled. This is a compatibility alias for `redpanda.key.subject.name.strategy` that determines how the topic and schema are mapped to a subject name in the Schema Registry.
826+
827+
*Type:* string
828+
829+
*Default:* `io.confluent.kafka.serializers.subject.TopicNameStrategy`
830+
831+
*Accepted values:* [`io.confluent.kafka.serializers.subject.TopicNameStrategy`, `io.confluent.kafka.serializers.subject.RecordNameStrategy`, `io.confluent.kafka.serializers.subject.TopicRecordNameStrategy`]
832+
833+
**Related topics**:
834+
835+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
836+
837+
---
838+
[[confluentvalueschemavalidation]]
839+
=== confluent.value.schema.validation
840+
841+
Enable validation of the schema ID for values on a record. This is a compatibility alias for `redpanda.value.schema.id.validation`. When enabled, Redpanda validates that the schema ID encoded in the record's value is registered in the Schema Registry according to the configured subject name strategy.
842+
843+
*Type:* boolean
844+
845+
*Default:* `false`
846+
847+
**Related topics**:
848+
849+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
850+
851+
---
852+
[[confluentvaluesubjectnamestrategy]]
853+
=== confluent.value.subject.name.strategy
854+
855+
The subject name strategy for values when `confluent.value.schema.validation` is enabled. This is a compatibility alias for `redpanda.value.subject.name.strategy`. This determines how the topic and schema are mapped to a subject name in the Schema Registry.
856+
857+
*Type:* string
858+
859+
*Default:* `io.confluent.kafka.serializers.subject.TopicNameStrategy`
860+
861+
*Accepted values:* [`io.confluent.kafka.serializers.subject.TopicNameStrategy`, `io.confluent.kafka.serializers.subject.RecordNameStrategy`, `io.confluent.kafka.serializers.subject.TopicRecordNameStrategy`]
862+
863+
**Related topics**:
864+
865+
- xref:manage:schema-reg/schema-id-validation.adoc[Server-Side Schema ID Validation]
866+
867+
---

0 commit comments

Comments
 (0)