Skip to content

docs (k8s): Move inline YAML examples to feature files for schema and topic controllers #1650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
david-yu wants to merge 3 commits into
base: main
Choose a base branch
from
Open

docs (k8s): Move inline YAML examples to feature files for schema and topic controllers #1650

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ad223a7
Move inline YAML examples to feature files for schema and topic contr…
david-yu Apr 1, 2026
adb7f1e
Add missing .yaml filename titles to schema controller examples
david-yu Apr 1, 2026
ada26e0
Add missing .yaml filename titles to topic YAML examples
david-yu Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions modules/manage/examples/kubernetes/schema-crds.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,72 @@ Feature: Schema CRDs
"""
And schema "order-event" is successfully synced
Then I should be able to check compatibility against "order-event" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage fully compatible schema (Avro)
Given there is no schema "fully-compatible-schema" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::full-compatibility-schema-manifest[]
# This manifest creates an Avro schema named "fully-compatible-schema" in the "basic" cluster.
# The schema uses Full compatibility, ensuring backward and forward compatibility across versions.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: fully-compatible-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Full
text: |
{
"type": "record",
"name": "ExampleRecord",
"fields": [
{ "type": "string", "name": "field1" },
{ "type": "int", "name": "field2" }
]
}
# end::full-compatibility-schema-manifest[]
"""
And schema "fully-compatible-schema" is successfully synced
Then I should be able to check compatibility against "fully-compatible-schema" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage order schema with references (Avro)
Given there is no schema "order-schema" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::schema-references-manifest[]
# This manifest creates an Avro schema named "order-schema" that references another schema.
# Schema references enable modular and reusable schema components for complex data structures.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: order-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
references:
- name: product-schema
subject: product
version: 1
text: |
{
"type": "record",
"name": "Order",
"fields": [
{ "name": "product", "type": "Product" }
]
}
# end::schema-references-manifest[]
"""
And schema "order-schema" is successfully synced
Then I should be able to check compatibility against "order-schema" in cluster "basic"
Comment on lines +142 to +175
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

❓ Verification inconclusive

Script executed:

#!/bin/bash
# Search for any existing schema definition with subject "product" in the feature files
rg -n "subject.*product" --type-add 'feature:*.feature' -t feature

# Check if there's a test setup that creates the product schema
rg -n "product-schema|subject: product" modules/manage/examples/kubernetes/

Repository: redpanda-data/docs

Repository: redpanda-data/docs
Exit code: 0

stdout:

modules/manage/examples/kubernetes/schema-crds.feature:162:          subject: product
modules/manage/examples/kubernetes/schema-crds.feature:161:        - name: product-schema
modules/manage/examples/kubernetes/schema-crds.feature:162:          subject: product

Script executed:

# Find the exact subject name of the product-catalog schema
rg -A 20 "product-catalog" modules/manage/examples/kubernetes/schema-crds.feature | head -40

# Search the entire repository for any schema with subject "product"
rg -n "subject.*product" --type-add 'feature:*.feature' -t feature

# Check if there's any test setup or Before hooks
rg -n "Background:|Before" modules/manage/examples/kubernetes/schema-crds.feature

# Look for other feature files that might set up schemas
fd -name "*.feature" modules/manage/examples/kubernetes/

Repository: redpanda-data/docs

Repository: redpanda-data/docs
Exit code: 2

stdout:

    Given there is no schema "product-catalog" in cluster "basic"
    When I apply Kubernetes manifest:
    """
# tag::product-catalog-protobuf-schema-manifest[]
    # This manifest creates a Protobuf schema named "product-catalog" in the "basic" cluster.
    # The schema defines a message "Product" with fields for product ID, name, price, and category.
    ---
    apiVersion: cluster.redpanda.com/v1alpha2
    kind: Schema
    metadata:
      name: product-catalog
    spec:
      cluster:
        clusterRef:
          name: basic
      schemaType: protobuf
      compatibilityLevel: Backward
      text: |
        syntax = "proto3";

        message Product {
          int32 product_id = 1;
          string product_name = 2;
          double price = 3;
          string category = 4;
        }
# end::product-catalog-protobuf-schema-manifest[]
    """
    And schema "product-catalog" is successfully synced
    Then I should be able to check compatibility against "product-catalog" in cluster "basic"

  `@skip`:gke `@skip`:aks `@skip`:eks
  Scenario: Manage order event schema (JSON)
    Given there is no schema "order-event" in cluster "basic"
    When I apply Kubernetes manifest:
    """
# tag::order-event-json-schema-manifest[]
    # This manifest creates a JSON schema named "order-event" in the "basic" cluster.
    # The schema requires an "order_id" (string) and a "total" (number) field, with no additional properties allowed.
    ---
modules/manage/examples/kubernetes/schema-crds.feature:162:          subject: product
3:  Background: Cluster available

stderr:

error: Found argument '-n' which wasn't expected, or isn't valid in this context

  If you tried to supply '-n' as a value rather than a flag, use '-- -n'

Usage: fd [OPTIONS] [pattern] [path]...

For more information try '--help'

Script executed:

# Read the full content of the feature file to understand structure and dependencies
cat -n modules/manage/examples/kubernetes/schema-crds.feature | grep -A 5 -B 5 "product-schema\|subject: product"

# Check if there's any documentation on how subjects are assigned
rg -i "subject.*derived\|schema.*subject\|metadata.*name" modules/manage/examples/kubernetes/ -A 2

# List all schemas defined in this feature file to see the pattern
rg "kind: Schema" -A 15 modules/manage/examples/kubernetes/schema-crds.feature | grep -E "name:|subject:|kind: Schema"

Repository: redpanda-data/docs

Repository: redpanda-data/docs
Exit code: 0

stdout:

   156	    spec:
   157	      cluster:
   158	        clusterRef:
   159	          name: basic
   160	      references:
   161	        - name: product-schema
   162	          subject: product
   163	          version: 1
   164	      text: |
   165	        {
   166	          "type": "record",
   167	          "name": "Order",
    kind: Schema
      name: customer-profile
          name: basic
    kind: Schema
      name: product-catalog
          name: basic
    kind: Schema
      name: order-event
          name: basic
    kind: Schema
      name: fully-compatible-schema
          name: basic
    kind: Schema
      name: order-schema
          name: basic
        - name: product-schema
          subject: product

Script executed:

# Check the Background section and overall structure of the feature file
head -20 modules/manage/examples/kubernetes/schema-crds.feature

# Search the entire repository for "product-schema" to see if it's created elsewhere
rg -i "product-schema" --type-add 'feature:*.feature' -t feature -B 3 -A 3

# Check for any reference to schema references in documentation or comments
rg -i "schema.*reference|reference.*schema" modules/manage/examples/kubernetes/ -B 2 -A 2

Repository: redpanda-data/docs

Repository: redpanda-data/docs
Exit code: 0

stdout:

`@cluster`:basic `@variant`:vectorized
Feature: Schema CRDs
  Background: Cluster available
    Given cluster "basic" is available

  `@skip`:gke `@skip`:aks `@skip`:eks
  Scenario: Manage customer profile schema (Avro)
    Given there is no schema "customer-profile" in cluster "basic"
    When I apply Kubernetes manifest:
    """
# tag::customer-profile-avro-schema-manifest[]
    # This manifest creates an Avro schema named "customer-profile" in the "basic" cluster.
    # The schema defines a record with fields for customer ID, name, and age.
    ---
    apiVersion: cluster.redpanda.com/v1alpha2
    kind: Schema
    metadata:
      name: customer-profile
    spec:
      cluster:
modules/manage/examples/kubernetes/schema-crds.feature-        clusterRef:
modules/manage/examples/kubernetes/schema-crds.feature-          name: basic
modules/manage/examples/kubernetes/schema-crds.feature-      references:
modules/manage/examples/kubernetes/schema-crds.feature:        - name: product-schema
modules/manage/examples/kubernetes/schema-crds.feature-          subject: product
modules/manage/examples/kubernetes/schema-crds.feature-          version: 1
modules/manage/examples/kubernetes/schema-crds.feature-      text: |
modules/manage/examples/kubernetes/schema-crds.feature-
modules/manage/examples/kubernetes/schema-crds.feature-  `@skip`:gke `@skip`:aks `@skip`:eks
modules/manage/examples/kubernetes/schema-crds.feature:  Scenario: Manage order schema with references (Avro)
modules/manage/examples/kubernetes/schema-crds.feature-    Given there is no schema "order-schema" in cluster "basic"
modules/manage/examples/kubernetes/schema-crds.feature-    When I apply Kubernetes manifest:
modules/manage/examples/kubernetes/schema-crds.feature-    """
modules/manage/examples/kubernetes/schema-crds.feature:# tag::schema-references-manifest[]
modules/manage/examples/kubernetes/schema-crds.feature:    # This manifest creates an Avro schema named "order-schema" that references another schema.
modules/manage/examples/kubernetes/schema-crds.feature:    # Schema references enable modular and reusable schema components for complex data structures.
modules/manage/examples/kubernetes/schema-crds.feature-    ---
modules/manage/examples/kubernetes/schema-crds.feature-    apiVersion: cluster.redpanda.com/v1alpha2
--
modules/manage/examples/kubernetes/schema-crds.feature-          ]
modules/manage/examples/kubernetes/schema-crds.feature-        }
modules/manage/examples/kubernetes/schema-crds.feature:# end::schema-references-manifest[]
modules/manage/examples/kubernetes/schema-crds.feature-    """
modules/manage/examples/kubernetes/schema-crds.feature-    And schema "order-schema" is successfully synced

Test scenario references a non-existent schema dependency.

The order-schema scenario declares a reference to product-schema with subject: product and version: 1:

references:
  - name: product-schema
    subject: product
    version: 1

However, no schema named product-schema (or with subject product) is created anywhere in this feature file or the test suite. The scenario includes assertions expecting successful sync and compatibility checks, which will fail because the referenced schema cannot be resolved.

Add a preceding scenario that creates a product-schema, or adjust the reference to point to an existing schema defined in this file (e.g., product-catalog).

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@modules/manage/examples/kubernetes/schema-crds.feature` around lines 142 -
175, The scenario for Schema named "order-schema" references a non-existent
dependency "product-schema" (subject: "product", version: 1), causing
sync/compatibility to fail; fix by either adding a preceding scenario that
creates the referenced schema (e.g., create a Schema resource named
"product-schema" with subject "product" and version 1 and ensure it's synced
before the order-schema scenario) or change the references block in the
"order-schema" manifest to point to an existing schema in this feature (for
example replace name/subject with the existing "product-catalog" schema and its
correct subject/version) so the reference can be resolved at test time.

52 changes: 52 additions & 0 deletions modules/manage/examples/kubernetes/topic-crds.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,55 @@ Feature: Topic CRDs
"""
And topic "topic1" is successfully synced
Then I should be able to produce and consume from "topic1" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage topic with write caching
Given there is no topic "chat-room" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::write-caching-topic-example[]
# This manifest creates a topic called "chat-room" with write caching enabled.
# Write caching provides better performance at the expense of durability.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: chat-room
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
write.caching: "true"
# end::write-caching-topic-example[]
"""
And topic "chat-room" is successfully synced
Then I should be able to produce and consume from "chat-room" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage topic with cleanup policy
Given there is no topic "compacted-topic" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::cleanup-policy-topic-example[]
# This manifest creates a topic with the cleanup policy set to "delete".
# The cleanup policy determines how partition log files are managed when they reach a certain size.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: compacted-topic
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
cleanup.policy: "delete"
# end::cleanup-policy-topic-example[]
"""
And topic "compacted-topic" is successfully synced
Then I should be able to produce and consume from "compacted-topic" in cluster "basic"
Comment on lines +55 to +79
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Topic name compacted-topic is misleading given cleanup.policy: "delete".

The scenario creates a topic named compacted-topic but sets cleanup.policy: "delete". This is confusing because:

  • compacted-topic suggests log compaction (cleanup.policy: "compact")
  • The actual policy is "delete", which deletes data based on retention settings

Consider renaming the topic to something like retained-topic or delete-policy-topic to match its configuration, or change the policy to "compact" if the intent was to demonstrate compaction.

Proposed fix (option 1: rename topic) 
   `@skip`:gke `@skip`:aks `@skip`:eks
   Scenario: Manage topic with cleanup policy
-    Given there is no topic "compacted-topic" in cluster "basic"
+    Given there is no topic "delete-policy-topic" in cluster "basic"
     When I apply Kubernetes manifest:
     """
 # tag::cleanup-policy-topic-example[]
     # This manifest creates a topic with the cleanup policy set to "delete".
     # The cleanup policy determines how partition log files are managed when they reach a certain size.
     ---
     apiVersion: cluster.redpanda.com/v1alpha2
     kind: Topic
     metadata:
-      name: compacted-topic
+      name: delete-policy-topic
     spec:
       cluster:
         clusterRef:
           name: basic
       partitions: 3
       replicationFactor: 1
       additionalConfig:
         cleanup.policy: "delete"
 # end::cleanup-policy-topic-example[]
     """
-    And topic "compacted-topic" is successfully synced
-    Then I should be able to produce and consume from "compacted-topic" in cluster "basic"
+    And topic "delete-policy-topic" is successfully synced
+    Then I should be able to produce and consume from "delete-policy-topic" in cluster "basic"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
@skip:gke @skip:aks @skip:eks
Scenario: Manage topic with cleanup policy
Given there is no topic "compacted-topic" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::cleanup-policy-topic-example[]
# This manifest creates a topic with the cleanup policy set to "delete".
# The cleanup policy determines how partition log files are managed when they reach a certain size.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: compacted-topic
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
cleanup.policy: "delete"
# end::cleanup-policy-topic-example[]
"""
And topic "compacted-topic" is successfully synced
Then I should be able to produce and consume from "compacted-topic" in cluster "basic"
`@skip`:gke `@skip`:aks `@skip`:eks
Scenario: Manage topic with cleanup policy
Given there is no topic "delete-policy-topic" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::cleanup-policy-topic-example[]
# This manifest creates a topic with the cleanup policy set to "delete".
# The cleanup policy determines how partition log files are managed when they reach a certain size.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: delete-policy-topic
spec:
cluster:
clusterRef:
name: basic
partitions: 3
replicationFactor: 1
additionalConfig:
cleanup.policy: "delete"
# end::cleanup-policy-topic-example[]
"""
And topic "delete-policy-topic" is successfully synced
Then I should be able to produce and consume from "delete-policy-topic" in cluster "basic"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@modules/manage/examples/kubernetes/topic-crds.feature` around lines 55 - 79,
The topic name "compacted-topic" conflicts with its configured cleanup.policy of
"delete"; fix this by either renaming the Topic metadata.name from
"compacted-topic" to a name matching the delete semantics (e.g.,
"retained-topic" or "delete-policy-topic") or change
spec.additionalConfig.cleanup.policy from "delete" to "compact" so the name and
policy align; update the Scenario text and any referenced steps that mention
"compacted-topic" to the new name to keep the test consistent.

35 changes: 7 additions & 28 deletions modules/manage/pages/kubernetes/k-manage-topics.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ The Redpanda Operator automatically configures the Topic controller to connect t

The `clusterRef` method automatically discovers connection details from the referenced Redpanda resource:

.`topic.yaml`
[,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand All @@ -138,6 +139,7 @@ spec:

The `staticConfiguration` method requires manually specifying all connection details:

.`topic.yaml`
[,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand Down Expand Up @@ -220,6 +222,7 @@ In addition to the general prerequisites for managing topics, you must have the

To create an Iceberg topic, set the `redpanda.iceberg.mode` configuration in the `additionalConfig` property of the `Topic` resource.

.`iceberg-topic.yaml`
[source,yaml]
----
apiVersion: cluster.redpanda.com/v1alpha2
Expand Down Expand Up @@ -326,21 +329,9 @@ With `write_caching` enabled at the cluster level, Redpanda fsyncs to disk accor
To override the cluster-level setting at the topic level, set the topic-level property xref:reference:topic-properties.adoc#writecaching[`write.caching`]:

.`example-topic.yaml`
[,yaml,lines=9]
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: chat-room
namespace: <namespace>
spec:
cluster:
clusterRef:
name: <cluster-name>
partitions: 3
replicationFactor: 3
additionalConfig:
write.caching: true
include::manage:example$kubernetes/topic-crds.feature[tags=write-caching-topic-example,indent=0]
----

With `write.caching` enabled at the topic level, Redpanda fsyncs to disk according to xref:reference:topic-properties.adoc#flushms[`flush.ms`] and xref:reference:topic-properties.adoc#flushbytes[`flush.bytes`], whichever is reached first.
Expand Down Expand Up @@ -627,21 +618,9 @@ CAUTION: Do not use `rpk` or any other Kafka clients to edit topics that you cre
The following example changes the cleanup policy for a topic:

.`example-topic.yaml`
[,yaml,lines=8-9]
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: <topic-name>
namespace: <namespace>
spec:
cluster:
clusterRef:
name: <cluster-name>
partitions: 3
replicationFactor: 3
additionalConfig:
cleanup.policy: "delete"
include::manage:example$kubernetes/topic-crds.feature[tags=cleanup-policy-topic-example,indent=0]
----

[,bash]
Expand Down
49 changes: 6 additions & 43 deletions modules/manage/pages/kubernetes/k-schema-controller.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,28 +172,10 @@ Compatibility modes determine how schema versions within a subject can evolve wi

For example, to set full compatibility, configure the Schema resource with:

[source,yaml]
.`schema.yaml`
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: fully-compatible-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Full
text: |
{
"type": "record",
"name": "ExampleRecord",
"fields": [
{ "type": "string", "name": "field1" },
{ "type": "int", "name": "field2" }
]
}
include::manage:example$kubernetes/schema-crds.feature[tags=full-compatibility-schema-manifest,indent=0]
----

Compatibility settings are essential for maintaining data consistency, especially when updating schemas over time.
Expand All @@ -206,29 +188,10 @@ NOTE: This feature is supported for Avro and Protobuf schemas.

Define a schema reference using the `references` field. The reference includes the name, subject, and version of the referenced schema:

[source,yaml]
.`schema.yaml`
[,yaml,indent=0]
----
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: order-schema
namespace: redpanda
spec:
cluster:
clusterRef:
name: basic
references:
- name: product-schema
subject: product
version: 1
text: |
{
"type": "record",
"name": "Order",
"fields": [
{ "name": "product", "type": "Product" }
]
}
include::manage:example$kubernetes/schema-crds.feature[tags=schema-references-manifest,indent=0]
----

== Update a schema
Expand Down
4 changes: 2 additions & 2 deletions ...les/manage/pages/kubernetes/security/authentication/k-schema-registry-acls.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,14 @@ In this example, a RedpandaRole called `read-only-role` is created in the cluste

The Group resource supports Schema Registry ACLs for OIDC groups.

In this example, ACLs are created for an OIDC group called `engineering` in the cluster called `sasl`. The authorization rules grant `Read` and `Describe` access to all topics with names starting with `team-` using a `prefixed` pattern type, and the same `Read` and `Describe` access to Schema Registry subjects matching the same prefix.

.`group-with-sr-acls.yaml`
[,yaml,indent=0]
----
include::manage:example$kubernetes/group-crds.feature[tags=manage-group-acls,indent=0]
----

In this example, ACLs are created for an OIDC group called `engineering` in the cluster called `sasl`. The authorization rules grant `Read` and `Describe` access to all topics with names starting with `team-` using a `prefixed` pattern type, and the same `Read` and `Describe` access to Schema Registry subjects matching the same prefix.

== Common use cases

The following examples show common patterns for configuring Schema Registry ACLs using the User resource.
Expand Down
Loading