| title | vastde pipelines |
|---|---|
| description | Manage VAST DataEngine pipelines |
Manage VAST DataEngine pipelines
Manage VAST DataEngine pipelines for orchestrating data processing workflows.
Pipelines are declarative workflows that define how data flows through your DataEngine application. A pipeline consists of multiple stages, each executing a function to transform, process, or route data. Pipelines enable you to build complex data processing workflows by chaining functions together.
Key concepts:
- Stages: Individual steps in the pipeline, each referencing a function
- Data Flow: How data moves from one stage to the next
- Error Handling: Dead letter queues and retry policies for failed stages
- Deployment: Pipelines must be deployed to become active
Pipelines can be defined in YAML configuration files that specify the stages, functions, and data flow. The deploy command activates a pipeline and begins processing data. Pipelines can be updated to modify the workflow without recreating them.
- create - Create a VAST DataEngine pipeline
- delete - Delete a VAST DataEngine pipline
- deploy - Deploy a pipeline to kubernetes
- get - Get a VAST DataEngine pipeline details or manifest
- list - List VAST DataEngine pipelines
- update - Update a VAST DataEngine pipeline
| Flag | Type | Description | Default |
|---|---|---|---|
--dry-run |
bool | Simulate the operation without making actual changes to the system | |
-o, --output |
string | Output format: json, yaml, human |
human |
--silent |
bool | Suppress UI outputs, such as spinner and success messages | |
-v, --verbose |
int | Verbosity level (0-9): 0=standard, 1=verbose, 2=detailed, 3=extended, 4=debug, 5=trace | 0 |
- vastde - VAST DataEngine CLI