|
| 1 | +# Aspire Pipelines Reference |
| 2 | + |
| 3 | +## Overview |
| 4 | +Aspire uses a pipeline-based deployment system that enables extensible, composable deployment workflows. Pipelines break deployment into discrete, well-defined steps that can be optimized for performance and reliability while maintaining clear visibility into the deployment process. |
| 5 | + |
| 6 | +### Why pipelines matter |
| 7 | +- **Intelligent concurrency:** Independent operations run in parallel, maximizing resource utilization and minimizing total deployment time. |
| 8 | +- **Granular control:** Each step can be executed, monitored, and retried independently with specific error reporting and targeted recovery capabilities. |
| 9 | +- **Smart dependency management:** Steps wait only for their actual dependencies, ensuring optimal execution order while maintaining correctness. |
| 10 | +- **Comprehensive observability:** Real-time progress reporting at both step and sub-task levels provides clear visibility into deployment operations. |
| 11 | +- **Flexible execution:** Run complete deployments or selective step execution based on your specific needs. |
| 12 | + |
| 13 | +### Core capabilities |
| 14 | +- **Extend deployments:** Add custom deployment steps for specific resources or scenarios. |
| 15 | +- **Control execution order:** Declare dependencies between steps to ensure proper sequencing. |
| 16 | +- **Run selective steps:** Execute only specific steps and their dependencies for faster iterations. |
| 17 | +- **Parallelize operations:** Independent steps can run concurrently for improved performance. |
| 18 | +- **State persistence:** Cache deployment state and parameters across multiple deployment runs. |
| 19 | +- **Intelligent granularity:** Balance between fine-grained control and deployment efficiency. |
| 20 | + |
| 21 | +## Core Concepts |
| 22 | + |
| 23 | +### Pipeline Steps |
| 24 | +A pipeline step represents a discrete unit of work in the application pipeline. Each step encapsulates a specific aspect of the deployment process, making the overall operation more manageable and debuggable. |
| 25 | + |
| 26 | +Each step has: |
| 27 | +- **Name:** A unique identifier for the step |
| 28 | +- **Action:** The work to be performed during deployment |
| 29 | +- **Dependencies:** Other steps that must complete before this step runs |
| 30 | +- **Resource context:** Access to the application’s resource model and configuration |
| 31 | + |
| 32 | +#### Example pipeline steps |
| 33 | +**Infrastructure steps (can run in parallel):** |
| 34 | +- `provision-cosmosdb`: Provision the CosmosDB database |
| 35 | +- `provision-storage`: Provision Azure Storage account |
| 36 | +- `provision-aca-env`: Provision Azure Container Apps environment |
| 37 | +- `provision-container-registry`: Set up the container image registry |
| 38 | + |
| 39 | +**Build steps (can run in parallel):** |
| 40 | +- `build-apiservice`: Build container image for the API service |
| 41 | +- `build-webfrontend`: Build container image for the frontend |
| 42 | + |
| 43 | +**Deployment steps (depend on builds and infrastructure):** |
| 44 | +- `push-apiservice`: Push API service image to registry |
| 45 | +- `push-webfrontend`: Push frontend image to registry |
| 46 | +- `deploy-apiservice`: Deploy API service to compute platform |
| 47 | +- `deploy-webfrontend`: Deploy frontend to compute platform |
| 48 | + |
| 49 | +### Step Execution Phases |
| 50 | +The pipeline system operates through three distinct phases: |
| 51 | +1. **Registration phase:** Steps are collected from resource annotations and pipeline API calls. |
| 52 | +2. **Resolution phase:** The system validates dependencies, calculates execution order, and identifies concurrency opportunities. |
| 53 | +3. **Execution phase:** Steps run according to the resolved plan, with parallel execution, dependency waiting, progress reporting, and error isolation. |
| 54 | + |
| 55 | +### Dependency Management |
| 56 | +Dependencies in Aspire pipelines are explicit and purpose-driven. Steps declare their relationships using: |
| 57 | +- `PipelineStep.DependsOn`: This step must run after another step completes successfully |
| 58 | +- `PipelineStep.RequiredBy`: Another step must run after this step (inverse dependency declaration) |
| 59 | + |
| 60 | +The pipeline system validates dependencies to ensure: |
| 61 | +- No circular dependencies |
| 62 | +- All references exist |
| 63 | +- Logical consistency |
| 64 | + |
| 65 | +### Optimal Granularity Design |
| 66 | +Resource-level granularity: Each infrastructure resource is provisioned as a separate step, enabling precise error reporting, targeted retry, progress transparency, and flexible recovery. |
| 67 | +Operation-level organization: Related operations are grouped logically while maintaining independence (e.g., build, push, deploy). |
| 68 | + |
| 69 | +## Well-known Steps |
| 70 | + |
| 71 | +### Entry Point Steps |
| 72 | +- `WellKnownSteps.Deploy`: Orchestrates the complete deployment process. |
| 73 | +- `WellKnownSteps.Publish`: Generates deployment artifacts without executing the actual deployment. |
| 74 | +- `WellKnownSteps.Build`: Builds container images for compute resources. |
| 75 | +- `WellKnownSteps.Push`: Pushes container images to registries. |
| 76 | + |
| 77 | +### Prerequisite Steps |
| 78 | +- `WellKnownSteps.BuildPrereq`: Pre-requisites for building (dependency resolution, environment setup). |
| 79 | +- `WellKnownSteps.DeployPrereqs`: Pre-requisites for deployment (authentication, environment validation). |
| 80 | +- `WellKnownSteps.PublishPrereqs`: Pre-requisites for publishing (build environment setup). |
| 81 | +- `WellKnownSteps.PushPrereq`: Pre-requisites for pushing images (registry authentication). |
| 82 | + |
| 83 | +### Resource-contributed Steps |
| 84 | +Resources can contribute their own pipeline steps through annotations. For example, a static site resource might contribute steps for building, configuring storage permissions, uploading files, and configuring a CDN. |
| 85 | + |
| 86 | +### Application-specific Steps |
| 87 | +Applications can add custom steps directly through the pipeline API for scenarios like database migrations, smoke tests, configuration updates, and notification workflows. |
| 88 | + |
| 89 | +## Execution Control |
| 90 | + |
| 91 | +### Running Specific Steps |
| 92 | +The `aspire do` command allows you to execute individual steps and their dependencies, providing fine-grained control over the deployment process. Use cases include incremental deployments, troubleshooting, development workflows, and selective operations. |
| 93 | + |
| 94 | +### Parallel Execution and Performance Optimization |
| 95 | +The pipeline system maximizes deployment performance through intelligent parallel execution, resource grouping, critical path optimization, and resource-aware scheduling. It provides robust failure handling, error isolation, and comprehensive logging for efficient debugging and recovery. |
| 96 | + |
| 97 | +--- |
| 98 | +For more, see the official Aspire documentation for pipelines. |
0 commit comments