Skip to content

Commit effc8f3

Browse files
leto-bbqleto-bbq
authored
Add ALTER PIPE docs to data sync manuals (#1140)
1 parent c1f044b commit effc8f3

16 files changed

Lines changed: 418 additions & 18 deletions

File tree

‎src/UserGuide/Master/Table/User-Manual/Data-Sync_apache.md‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,32 @@ IoTDB> show pipe alldatapipe_realtime
214214
```
215215

216216

217-
### 2.6 Synchronization Plugins
217+
### 2.6 Modify a Task
218+
219+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
220+
221+
```SQL
222+
ALTER PIPE [IF EXISTS] <PipeId>
223+
MODIFY/REPLACE SOURCE(...)
224+
MODIFY/REPLACE PROCESSOR(...)
225+
MODIFY/REPLACE SINK(...)
226+
```
227+
228+
Description:
229+
230+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
231+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
232+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
233+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
234+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
235+
236+
Example:
237+
238+
```SQL
239+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
240+
```
241+
242+
### 2.7 Synchronization Plugins
218243

219244
To make the architecture more flexible and adaptable to different synchronization scenarios, IoTDB supports plugin assembly in the synchronization task framework. The system provides some common pre-installed plugins, and you can also customize `processor` and `sink` plugins and load them into the IoTDB system.
220245

@@ -564,4 +589,4 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
564589
| load-tsfile-strategy | When synchronizing file data, ​​whether the receiver waits for the local load tsfile operation to complete before responding to the sender​​:<br>​​sync​​: Wait for the local load tsfile operation to complete before returning the response.<br>​​async​​: Do not wait for the local load tsfile operation to complete; return the response immediately. | String: sync / async | No | sync |
565590
| ssl.trust-store-path | Path to the trust store certificate for SSL connection. | String.Example: '127.0.0.1:6667,127.0.0.1:6668,127.0.0.1:6669', '127.0.0.1:6667' | Yes | - |
566591
| ssl.trust-store-pwd | Password for the trust store certificate. | Integer | Yes | - |
567-
| format | The payload formats for data transmission include the following options:<br> - hybrid: The format depends on what is passed from the processor (either tsfile or tablet), and the sink performs no conversion.<br> - tsfile: Data is forcibly converted to tsfile format before transmission. This is suitable for scenarios like data file backup.<br> - tablet: Data is forcibly converted to tsfile format before transmission. This is useful for data synchronization when the sender and receiver have incompatible data types (to minimize errors). | String: hybrid / tsfile / tablet | No | hybrid |
592+
| format | The payload formats for data transmission include the following options:<br> - hybrid: The format depends on what is passed from the processor (either tsfile or tablet), and the sink performs no conversion.<br> - tsfile: Data is forcibly converted to tsfile format before transmission. This is suitable for scenarios like data file backup.<br> - tablet: Data is forcibly converted to tsfile format before transmission. This is useful for data synchronization when the sender and receiver have incompatible data types (to minimize errors). | String: hybrid / tsfile / tablet | No | hybrid |

‎src/UserGuide/Master/Table/User-Manual/Data-Sync_timecho.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -213,7 +213,32 @@ IoTDB> show pipe alldatapipe_realtime
213213
```
214214

215215

216-
### 2.6 Synchronization Plugins
216+
### 2.6 Modify a Task
217+
218+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
219+
220+
```SQL
221+
ALTER PIPE [IF EXISTS] <PipeId>
222+
MODIFY/REPLACE SOURCE(...)
223+
MODIFY/REPLACE PROCESSOR(...)
224+
MODIFY/REPLACE SINK(...)
225+
```
226+
227+
Description:
228+
229+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
230+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
231+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
232+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
233+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
234+
235+
Example:
236+
237+
```SQL
238+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
239+
```
240+
241+
### 2.7 Synchronization Plugins
217242

218243
To make the architecture more flexible and adaptable to different synchronization scenarios, IoTDB supports plugin assembly in the synchronization task framework. The system provides some common pre-installed plugins, and you can also customize `processor` and `sink` plugins and load them into the IoTDB system.
219244

‎src/UserGuide/Master/Tree/User-Manual/Data-Sync_apache.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,32 @@ IoTDB> show pipe alldatapipe_realtime
234234
+--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
235235
```
236236

237-
### 2.6 Synchronization Plugins
237+
### 2.6 Modify Task
238+
239+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
240+
241+
```SQL
242+
ALTER PIPE [IF EXISTS] <PipeId>
243+
MODIFY/REPLACE SOURCE(...)
244+
MODIFY/REPLACE PROCESSOR(...)
245+
MODIFY/REPLACE SINK(...)
246+
```
247+
248+
Description:
249+
250+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
251+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
252+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
253+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
254+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
255+
256+
Example:
257+
258+
```SQL
259+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
260+
```
261+
262+
### 2.7 Synchronization Plugins
238263

239264
To make the overall architecture more flexible to match different synchronization scenario requirements, we support plugin assembly within the synchronization task framework. The system comes with some pre-installed common plugins that you can use directly. At the same time, you can also customize processor plugins and Sink plugins, and load them into the IoTDB system for use. You can view the plugins in the system (including custom and built-in plugins) with the following statement:
240265

‎src/UserGuide/Master/Tree/User-Manual/Data-Sync_timecho.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,32 @@ IoTDB> show pipe alldatapipe_realtime
236236
```
237237

238238

239-
### 2.6 Synchronization Plugins
239+
### 2.6 Modify Task
240+
241+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
242+
243+
```SQL
244+
ALTER PIPE [IF EXISTS] <PipeId>
245+
MODIFY/REPLACE SOURCE(...)
246+
MODIFY/REPLACE PROCESSOR(...)
247+
MODIFY/REPLACE SINK(...)
248+
```
249+
250+
Description:
251+
252+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
253+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
254+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
255+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
256+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
257+
258+
Example:
259+
260+
```SQL
261+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
262+
```
263+
264+
### 2.7 Synchronization Plugins
240265

241266
To make the overall architecture more flexible to match different synchronization scenario requirements, we support plugin assembly within the synchronization task framework. The system comes with some pre-installed common plugins that you can use directly. At the same time, you can also customize processor plugins and Sink plugins, and load them into the IoTDB system for use. You can view the plugins in the system (including custom and built-in plugins) with the following statement:
242267

‎src/UserGuide/latest-Table/User-Manual/Data-Sync_apache.md‎

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,32 @@ IoTDB> show pipe alldatapipe_realtime
214214
```
215215

216216

217-
### 2.6 Synchronization Plugins
217+
### 2.6 Modify a Task
218+
219+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
220+
221+
```SQL
222+
ALTER PIPE [IF EXISTS] <PipeId>
223+
MODIFY/REPLACE SOURCE(...)
224+
MODIFY/REPLACE PROCESSOR(...)
225+
MODIFY/REPLACE SINK(...)
226+
```
227+
228+
Description:
229+
230+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
231+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
232+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
233+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
234+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
235+
236+
Example:
237+
238+
```SQL
239+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
240+
```
241+
242+
### 2.7 Synchronization Plugins
218243

219244
To make the architecture more flexible and adaptable to different synchronization scenarios, IoTDB supports plugin assembly in the synchronization task framework. The system provides some common pre-installed plugins, and you can also customize `processor` and `sink` plugins and load them into the IoTDB system.
220245

@@ -564,4 +589,4 @@ pipe_all_sinks_rate_limit_bytes_per_second=-1
564589
| load-tsfile-strategy | When synchronizing file data, ​​whether the receiver waits for the local load tsfile operation to complete before responding to the sender​​:<br>​​sync​​: Wait for the local load tsfile operation to complete before returning the response.<br>​​async​​: Do not wait for the local load tsfile operation to complete; return the response immediately. | String: sync / async | No | sync |
565590
| ssl.trust-store-path | Path to the trust store certificate for SSL connection. | String.Example: '127.0.0.1:6667,127.0.0.1:6668,127.0.0.1:6669', '127.0.0.1:6667' | Yes | - |
566591
| ssl.trust-store-pwd | Password for the trust store certificate. | Integer | Yes | - |
567-
| format | The payload formats for data transmission include the following options:<br> - hybrid: The format depends on what is passed from the processor (either tsfile or tablet), and the sink performs no conversion.<br> - tsfile: Data is forcibly converted to tsfile format before transmission. This is suitable for scenarios like data file backup.<br> - tablet: Data is forcibly converted to tsfile format before transmission. This is useful for data synchronization when the sender and receiver have incompatible data types (to minimize errors). | String: hybrid / tsfile / tablet | No | hybrid |
592+
| format | The payload formats for data transmission include the following options:<br> - hybrid: The format depends on what is passed from the processor (either tsfile or tablet), and the sink performs no conversion.<br> - tsfile: Data is forcibly converted to tsfile format before transmission. This is suitable for scenarios like data file backup.<br> - tablet: Data is forcibly converted to tsfile format before transmission. This is useful for data synchronization when the sender and receiver have incompatible data types (to minimize errors). | String: hybrid / tsfile / tablet | No | hybrid |

‎src/UserGuide/latest-Table/User-Manual/Data-Sync_timecho.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -213,7 +213,32 @@ IoTDB> show pipe alldatapipe_realtime
213213
```
214214

215215

216-
### 2.6 Synchronization Plugins
216+
### 2.6 Modify a Task
217+
218+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
219+
220+
```SQL
221+
ALTER PIPE [IF EXISTS] <PipeId>
222+
MODIFY/REPLACE SOURCE(...)
223+
MODIFY/REPLACE PROCESSOR(...)
224+
MODIFY/REPLACE SINK(...)
225+
```
226+
227+
Description:
228+
229+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
230+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
231+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
232+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
233+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
234+
235+
Example:
236+
237+
```SQL
238+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
239+
```
240+
241+
### 2.7 Synchronization Plugins
217242

218243
To make the architecture more flexible and adaptable to different synchronization scenarios, IoTDB supports plugin assembly in the synchronization task framework. The system provides some common pre-installed plugins, and you can also customize `processor` and `sink` plugins and load them into the IoTDB system.
219244

‎src/UserGuide/latest/User-Manual/Data-Sync_apache.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -234,7 +234,32 @@ IoTDB> show pipe alldatapipe_realtime
234234
+--------------------+-----------------------+-------+---------------------------------------------------------------------------+-------------+--------------------------+----------------+-------------------+-------------------------+
235235
```
236236

237-
### 2.6 Synchronization Plugins
237+
### 2.6 Modify Task
238+
239+
The `ALTER PIPE` statement dynamically updates an existing PIPE and supports modifying or replacing the configuration of source, processor, and sink.
240+
241+
```SQL
242+
ALTER PIPE [IF EXISTS] <PipeId>
243+
MODIFY/REPLACE SOURCE(...)
244+
MODIFY/REPLACE PROCESSOR(...)
245+
MODIFY/REPLACE SINK(...)
246+
```
247+
248+
Description:
249+
250+
* Executing this operation does not change the running state of the PIPE. It is equivalent to keeping the processing progress of the original PipeId and creating a new PIPE at the original progress position.
251+
* The modify/replace parameters for source/processor/sink are all optional. If no modification parameter is specified, it is equivalent to deleting the current PIPE and recreating it with the original configuration and progress.
252+
* For a plugin specified with modify, the plugin's other parameters are retained, and only the given parameters are replaced or added.
253+
* For a plugin specified with replace, all parameters of the plugin are replaced directly.
254+
* When the [IF EXISTS] keyword is used, execution succeeds even if no Pipe with the same name exists, but no operation is actually performed.
255+
256+
Example:
257+
258+
```SQL
259+
ALTER PIPE A2B REPLACE SINK ('sink'='iotdb-thrift-sink', 'node-urls' = '127.0.0.1:6668');
260+
```
261+
262+
### 2.7 Synchronization Plugins
238263

239264
To make the overall architecture more flexible to match different synchronization scenario requirements, we support plugin assembly within the synchronization task framework. The system comes with some pre-installed common plugins that you can use directly. At the same time, you can also customize processor plugins and Sink plugins, and load them into the IoTDB system for use. You can view the plugins in the system (including custom and built-in plugins) with the following statement:
240265

0 commit comments

Comments
 (0)