Skip to content

Improve Deployment Ordering Documentation #746

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Dan Miller (milldr) merged 1 commit into from
Apr 24, 2025
Merged

Improve Deployment Ordering Documentation #746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/layers/accounts/account-baseline.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Deploy CloudTrail and ECR"
title: "Deploy CloudTrail and Account Budgets"
sidebar_label: "Deploy Account Baseline"
sidebar_position: 5
---
Expand All @@ -11,7 +11,7 @@ import StepNumber from '@site/src/components/StepNumber';
import AtmosWorkflow from '@site/src/components/AtmosWorkflow';

<Intro>
Now that all the accounts have been deployed, we need to finalize the setup of the accounts. This includes deploying CloudTrail and ECR. These foundational components will be necessary to move forward with the rest of the deployment.
Now that all the accounts have been deployed, we need to finalize the setup of the accounts. This includes deploying CloudTrail and optionally account budgets. These foundational components will be necessary to move forward with the rest of the deployment.
</Intro>

| Steps | Actions |
Expand All @@ -21,11 +21,11 @@ Now that all the accounts have been deployed, we need to finalize the setup of t

<Steps>
<Step>
## <StepNumber/> Deploy Baseline Components
## <StepNumber/> Deploy CloudTrail

Deploy CloudTrail and ECR with the following workflow:
Deploy CloudTrail and the the CloudTrail bucket:

<AtmosWorkflow workflow="deploy" fileName="baseline" />
<AtmosWorkflow workflow="deploy/cloudtrail" fileName="baseline" />
Copy link
Copy Markdown

@dukarc Cyrus (dukarc) Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI when we run this now Dan Miller (@milldr) we just tried running this .. fyi with our version of workflows delivered we don't have this workflow

 √ : [superadmin] (HOST) workspace ⨠ atmos workflow deploy/cloudtrail -f baseline
 Invalid Workflow Name 

No workflow exists with the name  deploy/cloudtrail

Copy link
Copy Markdown
Member Author

@milldr Dan Miller (milldr) Apr 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you wont have any workflow updates your version of workflows since they exist in your repo. This is all you need to add

dont forget to change the environment to your default

# stacks/workflows/baseline.yaml
       - command: workflow vendor -f baseline
       - command: workflow init/tfstate -f baseline
       - command: workflow deploy/tfstate -f baseline
-      - command: workflow deploy -f baseline
+      - command: workflow deploy/cloudtrail -f baseline
+      - command: workflow deploy/ecr -f baseline
 
   vendor:
     description: Vendor baseline layer components.
 @@ -36,9 +37,13 @@ workflows:
     steps:
       - command: terraform deploy tfstate-backend --stack core-use1-root
 
-  deploy:
+  deploy/cloudtrail:
     description: Start AWS Cloudtrail in audit and root accounts to track changes across the org.
     steps:
       - command: terraform deploy cloudtrail-bucket -s core-use1-audit
       - command: terraform deploy cloudtrail -s core-gbl-root
 
+  deploy/ecr:
+    description: Deploy ECR in the artifacts account to use as our container registry
+    steps:
       - command: terraform deploy ecr -s core-use1-artifacts
here's the whole thing 
workflows:
  all:
    description: run all workflows
    steps:
      - command: workflow vendor -f baseline
      - command: workflow init/tfstate -f baseline
      - command: workflow deploy/tfstate -f baseline
      - command: workflow deploy/cloudtrail -f baseline
      - command: workflow deploy/ecr -f baseline

  vendor:
    description: Vendor baseline layer components.
    steps:
      - command: vendor pull --component account
      - command: vendor pull --component account-map
      - command: vendor pull --component account-quotas
      - command: vendor pull --component account-settings
      - command: vendor pull --component cloudtrail
      - command: vendor pull --component cloudtrail-bucket
      - command: vendor pull --component ecr
      - command: vendor pull --component tfstate-backend

  init/tfstate:
    description: Provision Terraform State Backend for initial deployment.
    steps:
      - command: terraform deploy tfstate-backend -var=access_roles_enabled=false --stack core-use1-root --auto-generate-backend-file=false
      - command: until aws s3 ls acme-core-use1-root-tfstate; do sleep 5; done
        type: shell
      - command: terraform deploy tfstate-backend -var=access_roles_enabled=false --stack core-use1-root --init-run-reconfigure=false

  deploy/tfstate:
    description: >-
      Deploy Terraform State Backend.

      Finalize Terraform State Backend access, which requires AWS Teams to be deployed first.
    steps:
      - command: terraform deploy tfstate-backend --stack core-use1-root

  deploy/cloudtrail:
    description: Start AWS Cloudtrail in audit and root accounts to track changes across the org.
    steps:
      - command: terraform deploy cloudtrail-bucket -s core-use1-audit
      - command: terraform deploy cloudtrail -s core-gbl-root

  deploy/ecr:
    description: Deploy ECR in the artifacts account to use as our container registry
    steps:
      - command: terraform deploy ecr -s core-use1-artifacts

</Step>

<Step>
Expand Down
36 changes: 36 additions & 0 deletions docs/layers/containers/prerequisite.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
title: "Prepare Container Registry"
sidebar_label: "Prepare Container Registry"
sidebar_position: 10
---
import Intro from '@site/src/components/Intro';
import KeyPoints from '@site/src/components/KeyPoints';
import Steps from '@site/src/components/Steps';
import Step from '@site/src/components/Step';
import StepNumber from '@site/src/components/StepNumber';
import AtmosWorkflow from '@site/src/components/AtmosWorkflow';

<Intro>
Now that the GitHub OIDC Provider has been deployed, we can proceed with setting up the necessary prerequisites for containers. The first prerequisite is deploying Amazon Elastic Container Registry (ECR) repositories that will be used to store container images built by GitHub Actions workflows.
</Intro>

| Steps | Actions |
| -------------------------- | ----------------------------------- |
| Deploy ECR repositories | `atmos workflow deploy/ecr -f baseline` |

<Steps>
<Step>
## <StepNumber/> Deploy ECR Repositories

Deploy the ECR repositories that will be used by GitHub Actions workflows:

<AtmosWorkflow workflow="deploy/ecr" fileName="baseline" />

We use ECR for two main purposes:
<Steps>
1. Storing the Geodesic base image that provides the development environment and tooling
2. Storing container images built during CI steps of application release workflows
</Steps>

</Step>
</Steps>
69 changes: 58 additions & 11 deletions docs/layers/gitops/setup.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import StepNumber from '@site/src/components/StepNumber';
import TaskList from '@site/src/components/TaskList';
import Admonition from '@theme/Admonition';
import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
import CodeBlock from '@theme/CodeBlock';

## Quick Start

Expand Down Expand Up @@ -58,23 +59,36 @@ import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
<Step>
### <StepNumber/> Authentication Prerequisites

The GitHub Action workflows expect the `gitops` AWS Team to be properly setup and connected to GitHub OIDC. Both of
The GitHub Action workflows expect both the `gitops` and `planners` AWS Teams to be properly setup and connected to GitHub OIDC. Both of
these components should already be deployed with `aws-teams`/`aws-team-roles` and `github-oidc-provider` respectively,
but `github-oidc-provider` will likely need to deployed to several additional accounts. Verify the following to complete
the authentication prerequisites.

<Admonition type="info" title="Trusted GitHub Repos">

By default in the Reference Architecture, the `trusted_github_repos` input is commented out for `aws-teams`. Now is the time to uncomment those lines. Follows the tasks below. Please see `stacks/catalog/aws-teams.yaml`

</Admonition>

<TaskList>
- The `gitops` Team is defined and deployed by `aws-teams`.
- The `gitops` team adds a trusted relationship with the given infrastructure repo, via `trusted_github_repos`.
_Capitalization matters!_
- The `gitops` and `planners` Teams are defined and deployed by `aws-teams`.
- Both teams have trusted relationships with the infrastructure repo via `trusted_github_repos`.
_Capitalization matters!_ In the reference architecture, these values are initially commented out and will need to be updated with your specific repository information:
```yaml
trusted_github_repos:
gitops:
- "acme/infra-acme"
components:
terraform:
aws-teams:
vars:
trusted_github_repos:
gitops:
- "acme/infra:main"
planners:
- "acme/infra"
```
- The `aws-team-roles` default catalog allows the `gitops` team to assume the `terraform` role, including anywhere
`aws-team-roles` is overwritten (`plat-dev` and `plat-sandbox`)
- `tfstate-backend` allows `gitops` to assume the default access role from the `core-identity` account
- Similarly, the `planners` team can assume the `planner` role in `aws-team-roles` to plan Terraform only.
- `tfstate-backend` allows both teams to assume the default access role from the `core-identity` account
- `github-oidc-provider` is deployed to every account that GitHub will be able to access. This should be every account
except `root`.
- The workflows have adequate permission
Expand Down Expand Up @@ -119,14 +133,47 @@ import AtmosWorkflow from '@site/src/components/AtmosWorkflow';

<Step>

### <StepNumber/> Deploy
### <StepNumber/> Deploy GitOps Prerequisites

Deploy three components, `gitops/s3-bucket`, `gitops/dynamodb`, and `gitops` with the following workflow:
Deploy the GitOps prerequisite components, `gitops/s3-bucket`, `gitops/dynamodb`, and `gitops` with the following workflow

<AtmosWorkflow workflow="deploy/gitops" fileName="gitops" />

And that's it!
</Step>
<Step>
### <StepNumber/> Reapply `aws-teams`

Now we need to reapply `aws-teams` to add the trusted GitHub repositories to `gitops` and `planners`.

Uncomment or add the `trusted_github_repos` input:

```yaml
# stacks/catalog/aws-teams.yaml
components:
terraform:
aws-teams:
vars:
trusted_github_repos:
gitops:
- "acme/infra:main"
planners:
- "acme/infra"
```

Run the following command to apply:

<Admonition type="info" title="Use a Privileged Role">

`aws-teams` is a sensitive component deployed to the `core-identity` account and therefore needs to be applied with a role or user with access to the account. For example use the `managers` AWS Team or the SuperAdmin user.

</Admonition>

```bash
atmos terraform apply aws-teams -s core-gbl-identity
```

</Step>
And that's it! Now you can try creating a new pull request. If properly configured, you should see GitHub Actions kick off `Atmos Terraform Plan`.
</Steps>


Expand Down
42 changes: 23 additions & 19 deletions sidebars.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,13 +208,33 @@ module.exports = {
items: [
{
type: 'category',
label: 'Container Orchestration',
label: 'Configure GitHub Actions',
collapsible: true,
collapsed: true,
link: {
type: 'doc',
id: 'layers/github-actions/github-actions'
},
items: [
{
type: 'autogenerated',
dirName: 'layers/github-actions',
}
]
},
{
type: 'category',
label: 'Orchestrate Containers',
collapsible: true,
collapsed: true,
items: [
{
type: 'autogenerated',
dirName: 'layers/containers',
},
{
type: 'category',
label: 'Using ECS',
label: 'Use ECS',
collapsible: true,
collapsed: true,
link: {
Expand All @@ -230,7 +250,7 @@ module.exports = {
},
{
type: 'category',
label: 'Using EKS',
label: 'Use EKS',
collapsible: true,
collapsed: true,
link: {
Expand All @@ -246,22 +266,6 @@ module.exports = {
},
]
},
{
type: 'category',
label: 'Configure GitHub Actions',
collapsible: true,
collapsed: true,
link: {
type: 'doc',
id: 'layers/github-actions/github-actions'
},
items: [
{
type: 'autogenerated',
dirName: 'layers/github-actions',
}
]
},
{
type: 'category',
label: 'Automate Terraform',
Expand Down
Loading