Skip to content

Commit f83db51

Browse files
committed
remove old plugin reference
1 parent 3ef010c commit f83db51

3 files changed

Lines changed: 5 additions & 9831 deletions

File tree

  • docs

docs/build/kafka-consumer/index.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ By using the [cmem-plugin-kafka](https://pypi.org/project/cmem-plugin-kafka/) [P
1717

1818
## Installation
1919

20-
In order to use the Kafka Consumer workflow task, you need to extend your Corporate Memory instance with the `cmem-plugin-kafka` package.
20+
In order to use the [Kafka Consumer](../reference/customtask/cmem_plugin_kafka-ReceiveMessages.md) or [Kafka Producer](../reference/customtask/cmem_plugin_kafka-SendMessages.md) workflow task, you need to extend your Corporate Memory instance with the `cmem-plugin-kafka` package.
2121
This can be done by using cmemc:
2222

2323
```shell-session title="Installing cmem-plugin-kafka on the instance 'my-cmem'"
@@ -74,7 +74,7 @@ There are two main modes how the consumer handles received messages: either the
7474

7575
### Write Messages to a Dataset
7676

77-
In order to write the received messages to a dataset, the option **Messages Dataset** needs to be set. Only JSON and XML message formats are supported in this mode. So depending on the message format a [JSON](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md#json) or [XML Dataset](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md#xml) needs to be created and configured as the **Messages Dataset**.
77+
In order to write the received messages to a dataset, the option **Messages Dataset** needs to be set. Only JSON and XML message formats are supported in this mode. So depending on the message format a [JSON](../reference/dataset/json.md) or [XML Dataset](../reference/dataset/xml.md) needs to be created and configured as the **Messages Dataset**.
7878

7979
![Choose a dataset according to the message format](configure-message-dataset.png){ class="bordered" }<!-- 24.1 -->
8080

@@ -92,15 +92,15 @@ In the "message streaming mode" (**Messages Dataset** is not set) the received m
9292
- **ts-production** — the timestamp when the message was written to the topic,
9393
- **ts-consumption** — the timestamp when the message was consumed from the topic.
9494

95-
Connect the output of Kafka Consumer inside a Workflow to a tabular dataset (e.g. a [CSV Dataset](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md#csv)) or directly to a transformation task.
95+
Connect the output of Kafka Consumer inside a Workflow to a tabular dataset (e.g. a [CSV Dataset](../reference/dataset/csv.md)) or directly to a transformation task.
9696

9797
![](demo-wf-2.png){ class="bordered" }<!-- 24.1 -->
9898

99-
The message content is captured as plain text. In order to process complex message content, the `content` path needs to be parsed with operators such as [Parse JSON](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md#parse-json) or [Parse XML](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md#parse-xml) to process the message content in a transformation.
99+
The message content is captured as plain text. In order to process complex message content, the `content` path needs to be parsed with operators such as [Parse JSON](../reference/customtask/JsonParserOperator.md) or [Parse XML](../reference/customtask/XmlParserOperator.md) to process the message content in a transformation.
100100

101101
![](demo-wf-3.png){ class="bordered" }<!-- 24.1 -->
102102

103-
Any modifications to the message set, such as filtering, can be done prior to parsing the content. One could for example remove duplicates (according to the message key) from the messages by using the [Distinct-by task](../../deploy-and-configure/configuration/dataintegration/plugin-reference/index.md).
103+
Any modifications to the message set, such as filtering, can be done prior to parsing the content. One could for example remove duplicates (according to the message key) from the messages by using the [Distinct-by task](../reference/customtask/DistinctBy.md).
104104

105105
![](demo-wf-4.png){ class="bordered" }<!-- 24.1 -->
106106

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
nav:
22
- Build (DataIntegration): index.md
3-
- Plugin Reference: plugin-reference
43
- Activity Reference: activity-reference
54

0 commit comments

Comments
 (0)