Skip to content

Commit e84d246

Browse files
authored
DOC-1211 verifying auditing and wasm in cloud (#1079)
1 parent e1e8f59 commit e84d246

6 files changed

Lines changed: 50 additions & 97 deletions

File tree

modules/develop/pages/data-transforms/build.adoc

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,38 @@ endif::[]
1919
* For Rust projects, you must have the latest stable version of https://rustup.rs/[Rust^].
2020
* For JavaScript and TypeScript projects, you must have the https://nodejs.org/en/download/package-manager[latest long-term-support release of Node.js^].
2121

22+
ifdef::env-cloud[]
23+
== Enable data transforms
24+
25+
Data transforms is disabled on all clusters by default. Before you can deploy data transforms to a cluster, you must first enable the feature.
26+
27+
Use the Cloud API to set xref:ROOT:reference:properties/cluster-properties.adoc#data_transforms_enabled[`data_transforms_enabled`] to `true`.
28+
29+
[,bash]
30+
----
31+
# Store your cluster ID in a variable
32+
export RP_CLUSTER_ID=<cluster-id>
33+
34+
# Retrieve a Redpanda Cloud access token
35+
export RP_CLOUD_TOKEN=`curl -X POST "https://auth.prd.cloud.redpanda.com/oauth/token" \
36+
-H "content-type: application/x-www-form-urlencoded" \
37+
-d "grant_type=client_credentials" \
38+
-d "client_id=<client-id>" \
39+
-d "client_secret=<client-secret>"`
40+
41+
# Update cluster configuration to enable data transforms
42+
curl -H "Authorization: Bearer ${RP_CLOUD_TOKEN}" -X PATCH \
43+
"https://api.cloud.redpanda.com/v1/clusters/${RP_CLUSTER_ID}" \
44+
-H 'accept: application/json'\
45+
-H 'content-type: application/json' \
46+
-d '{"cluster_configuration":{"custom_properties": {"data_transforms_enabled":true}}}'
47+
----
48+
49+
The xref:api:ROOT:cloud-controlplane-api.adoc#patch-/v1/clusters/-cluster.id-[`PATCH /clusters/{cluster.id}`] request returns the ID of a long-running operation. The operation may take up to ten minutes to complete. You can check the status of the operation by polling the xref:api:ROOT:cloud-controlplane-api.adoc#get-/v1/operations/-id-[`GET /operations/\{id}`] endpoint.
50+
51+
NOTE: You must restart your cluster if you change this configuration for a running cluster.
52+
endif::[]
53+
2254
[[init]]
2355
== Initialize a data transforms project
2456

modules/develop/pages/data-transforms/how-transforms-work.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ This section outlines the limitations of data transforms. These constraints are
5454
=== General
5555

5656
- *No external access*: Transform functions have no external access to disk or network resources.
57-
- *Single record transforms*: Only single record transforms are supported, but multiple output records from a single input record are supported. For aggregations, joins, or complex transformations, consider xref:redpanda-connect:ROOT:about.adoc[Redpanda Connect] or https://flink.apache.org/[Apache Flink^].
57+
- *Single message transforms*: Only single record transforms are supported, but multiple output records from a single input record are supported. For aggregations, joins, or complex transformations, consider using xref:redpanda-connect:ROOT:about.adoc[Redpanda Connect] or https://flink.apache.org/[Apache Flink^].
5858
- *Output topic limit*: Up to eight output topics are supported.
5959
- *Delivery semantics*: Transform functions have at-least-once delivery.
6060
- *Transactions API*: When clients use the Kafka Transactions API on partitions of an input topic, transform functions process only committed records.

modules/develop/pages/data-transforms/run-transforms-index.adoc

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,4 @@
22
:description: Choose your deployment environment to get started with building and deploying your first transform function in Redpanda.
33
:page-aliases: reference:rpk/rpk-wasm/rpk-wasm.adoc, reference:rpk/rpk-wasm.adoc, reference:rpk/rpk-wasm/rpk-wasm-deploy.adoc, reference:rpk/rpk-wasm/rpk-wasm-generate.adoc, reference:rpk/rpk-wasm/rpk-wasm-remove.adoc, data-management:data-transform.adoc, labs:data-transform/index.adoc
44
:page-layout: index
5-
:page-categories: Development, Stream Processing, Data Transforms
6-
// tag::single-source[]
7-
8-
// end::single-source[]
5+
:page-categories: Development, Stream Processing, Data Transforms

modules/develop/pages/data-transforms/run-transforms.adoc

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,5 @@
22
:description: Learn how to build and deploy your first transform function in Linux deployments.
33
:page-context-links: [{"name": "Linux", "to": "develop:data-transforms/run-transforms.adoc" },{"name": "Kubernetes", "to": "develop:data-transforms/k-run-transforms.adoc" } ]
44
:page-categories: Development, Stream Processing, Data Transforms
5-
// tag::single-source[]
65

76
include::develop:partial$run-transforms.adoc[]
8-
9-
// end::single-source[]

modules/develop/partials/run-transforms.adoc

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,3 @@
1-
// tag::single-source[]
2-
31
Data transforms let you run common data streaming tasks, like filtering, scrubbing, and transcoding, within Redpanda. For example, you may have consumers that require you to redact credit card numbers or convert JSON to Avro. Data transforms can also interact with the Redpanda Schema Registry to work with encoded data types.
42

53
Data transforms use a WebAssembly (Wasm) engine inside a Redpanda broker. A Wasm function acts on a single record in an input topic. You can develop and manage data transforms with xref:reference:rpk/rpk-transform/rpk-transform.adoc[`rpk transform`] commands.
@@ -700,5 +698,3 @@ rpk transform delete data-transforms-tutorial --no-confirm
700698
- xref:reference:data-transforms/golang-sdk.adoc[]
701699
- xref:reference:data-transforms/rust-sdk.adoc[]
702700
- xref:reference:rpk/rpk-transform/rpk-transform.adoc[`rpk transform` commands]
703-
704-
// end::single-source[]

0 commit comments

Comments
 (0)