Skip to content

Commit 96e7269

Browse files
pmatsinopoulospmatsinopoulos
committed
Merge remote-tracking branch 'origin/fix-ami' into staging
2 parents e833d2c + 52e304d commit 96e7269

17 files changed

Lines changed: 249 additions & 70 deletions

‎.github/workflows/continuos_deployment_staging.yml‎

Lines changed: 6 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -71,29 +71,9 @@ jobs:
7171
unset AWS_PROFILE
7272
terraform plan -out planfile
7373
74-
# ---------------------
75-
- name: Terraform apply
76-
run: |
77-
cd terraform
78-
cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
79-
. ./.envrc.good
80-
cd ${{ env.ENVIRONMENT }}
81-
cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
82-
. ./.envrc.good
83-
unset AWS_PROFILE
84-
terraform apply -input=false planfile
85-
86-
# --------------------------------------------------------------------------------------
87-
# - name: Post Deploy - Test Service for Server has Correct Task Definition Deployed
88-
# run: |
89-
# cd terraform
90-
# cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
91-
# . ./.envrc.good
92-
# cd ${{ env.ENVIRONMENT }}
93-
# cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
94-
# . ./.envrc.good
95-
# unset AWS_PROFILE
96-
# export ECS_CLUSTER_NAME_SERVER=$(terraform output -raw error_logs_server_cluster_name)
97-
# export ECS_SERVICE_NAME_SERVER=$(terraform output -raw error_logs_server_service_name)
98-
# export ECS_TASK_DEFINITION_SERVER=$(terraform output -raw error_logs_server_task_definition_revision)
99-
# ../test_deployed_task_definitions.sh "${AWS_REGION}" "${ECS_CLUSTER_NAME_SERVER}" "${ECS_SERVICE_NAME_SERVER}" "${ECS_TASK_DEFINITION_SERVER}"
74+
# Upload the plan as an artifact
75+
- name: Upload Terraform plan
76+
uses: actions/upload-artifact@v3
77+
with:
78+
name: terraform-plan
79+
path: terraform/${{ env.ENVIRONMENT }}/planfile

‎.github/workflows/continuos_deployment_staging_apply_changes.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
name: Continuos Deployment Action Staging Apply Changes
2+
3+
on:
4+
workflow_dispatch:
5+
6+
env:
7+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID_PLAY_DEVOPS }}
8+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY_PLAY_DEVOPS }}
9+
10+
defaults:
11+
run:
12+
shell: bash
13+
14+
jobs:
15+
terraform_apply:
16+
runs-on: ubuntu-latest
17+
environment: staging
18+
19+
env:
20+
ETHEREUM_PEM: ${{ secrets.ETHEREUM_PEM }}
21+
ENVIRONMENT: staging
22+
GF_SECURITY_ADMIN_USER: "${{ secrets.GF_SECURITY_ADMIN_USER }}"
23+
GF_SECURITY_ADMIN_PASSWORD: "${{ secrets.GF_SECURITY_ADMIN_PASSWORD }}"
24+
PROMETHEUS_ADMIN_PASSWORD: "${{ secrets.PROMETHEUS_ADMIN_PASSWORD }}"
25+
PROMETHEUS_ADMIN_PASSWORD_BCRYPTED: "${{ secrets.PROMETHEUS_ADMIN_PASSWORD_BCRYPTED }}"
26+
27+
steps:
28+
- name: Checkout
29+
uses: actions/checkout@v3
30+
31+
- name: Download Terraform plan
32+
uses: actions/download-artifact@v3
33+
with:
34+
name: terraform-plan
35+
path: terraform/${{ env.ENVIRONMENT }}
36+
37+
- name: Terraform apply
38+
run: |
39+
cd terraform
40+
cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
41+
. ./.envrc.good
42+
cd ${{ env.ENVIRONMENT }}
43+
cat .envrc | grep -vE '^(dotenv_if_exists|source_env)' > .envrc.good
44+
. ./.envrc.good
45+
unset AWS_PROFILE
46+
terraform apply -input=false planfile

‎docs/ETHEREUM.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# WIP
2+
3+
Document is WIP
4+
5+
# Ethereum
6+
7+
This node is needed for the following chains:
8+
9+
- `mainnet`
10+
- `sepolia`
11+
- `base mainnet` because L2 chains require an integration with an L1 node
12+
- `base sepolia` because L2 chains require an integration with an L1 node
13+
14+
# Order of Setup Actions
15+
16+
## 1. AMI
17+
18+
We select the AMI to be an Ubuntu with EBS root device and EBS optimized.
19+
20+
See [../terraform/ethereum_ec2_instance.tf](../terraform/ethereum_ec2_instance.tf) resource `data.aws_ami.ubuntu`.
21+
22+
## 2. EC2 Instance
23+
24+
### User Data
25+
26+
#### Configure custom SSH port
27+
28+
### Systems Manager Agent
29+
30+
This is already installed on the AMI. So, there is no need to install.
31+
32+
However, there is need to configure.
33+
34+
- 1st it should have the correct privileges and have all its directories ready
35+
- 2nd it should be sending its logs to cloudwatch agent.
36+
37+
Does this mean that I need to have cloudwatch agent installed first?
38+
Maybe the correct order of things is:
39+
40+
41+
- prepare directories for ssm-agent
42+
- configure ssm agent to log in json format
43+
- restart ssm agent
44+
45+
46+
47+
Then at some point we prepare the cloud watch log groups for SSM agent. Does this action have any dependency. No it doesn't.
48+
49+
50+
51+
Then maybe cloudwatch agent should be the last step in the process.
52+
- install cloudwatch agent
53+
- configure cloudwatch agent to read all the services files we want to send over.
54+
This ^ needs to be done after all cloudwatch log groups have been prepared. So, there is a dependency to them.

‎terraform/bash/prepare_grafana_directories.sh‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
#!/bin/bash
2+
3+
set -e # e: exit if any command has a non-zero exit status
4+
set -x # x: all executed commands are printed to the terminal
5+
set -u # u: all references to variables that have not been previously defined cause an error
6+
7+
# The purpose of this script is to prepare the necessary directories
8+
# for Grafana. It should be executed after the second EBS volume
9+
# is attached to the EC2 Ethereum node instance and after the /data
10+
# folder is mounted.
11+
12+
USER=$1
13+
14+
sudo mkdir -p /data/grafana/data
15+
sudo mkdir -p /home/${USER}/grafana/datasources
16+
sudo mkdir -p /home/${USER}/grafana/dashboards
17+
sudo chown -R ${USER}:${USER} /home/${USER}/grafana
18+
19+
# On most Linux systems, the user ID 472 does not correspond to
20+
# named user default, which is why getent passwd 472 returns nothing.
21+
# This ID is commonly used by Grafana when running in Docker containers,
22+
# where the Grafana process inside the container runs as UID 472 and GID 0 (root).
23+
# On the host system, the ID may not have associated names, but setting
24+
# ownership to 472:root ensures that the containerized Grafana process can access
25+
# and modify the files, because it matches the UID/GID it runs as inside the container.
26+
sudo chown -R 472:root /data/grafana/data
27+
sudo chmod -R u+rwx,g+rx,g-w,o+rx,o-w /data/grafana/data

‎terraform/bash/prepare_prometheus_directories.sh‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/bin/bash
2+
3+
set -e # e: exit if any command has a non-zero exit status
4+
set -x # x: all executed commands are printed to the terminal
5+
set -u # u: all references to variables that have not been previously defined cause an error
6+
7+
# The purpose of this script is to prepare the necessary directories
8+
# for Prometheus. It should be executed after the second EBS volume
9+
# is attached to the EC2 Ethereum node instance and after the /data
10+
# folder is mounted.
11+
12+
USER=$1
13+
14+
# Create the "prometheus" folder
15+
sudo mkdir -p /home/${USER}/prometheus
16+
sudo chown ${USER}:${USER} /home/${USER}/prometheus
17+
18+
sudo mkdir -p /data/prometheus/data
19+
sudo chown -R nobody:nogroup /data/prometheus/data
20+
sudo chmod -R u+rwx,g+rx,g-w,o+rx,o-w /data/prometheus/data

‎terraform/ethereum_ec2_instance.tf‎

Lines changed: 1 addition & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,32 +1,5 @@
1-
data "aws_ami" "ubuntu" {
2-
most_recent = true
3-
4-
filter {
5-
name = "block-device-mapping.delete-on-termination"
6-
values = ["true"]
7-
}
8-
9-
filter {
10-
name = "is-public"
11-
values = ["true"]
12-
}
13-
14-
15-
filter {
16-
name = "name"
17-
values = ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-arm64-server*"]
18-
}
19-
20-
filter {
21-
name = "root-device-type"
22-
values = ["ebs"]
23-
}
24-
25-
owners = ["099720109477"] # Canonical
26-
}
27-
281
resource "aws_instance" "ethereum" {
29-
ami = data.aws_ami.ubuntu.id
2+
ami = var.ethereum_node.ami_id
303
associate_public_ip_address = var.ethereum_node.associate_public_ip_address
314
availability_zone = "${var.region}${var.ethereum_node.availability_zone}"
325
ebs_optimized = var.ethereum_node.ebs_optimized
@@ -68,7 +41,6 @@ resource "aws_instance" "ethereum" {
6841
})
6942
user_data_replace_on_change = false
7043

71-
7244
tags = {
7345
Name = "${var.project}-${var.environment}-ethereum"
7446
}

‎terraform/ethereum_prepare_grafana_directories.tf‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
locals {
2+
ethereum_prepare_grafana_directories_filename = "prepare_grafana_directories.sh"
3+
destination_ethereum_prepare_grafana_directories_filename = "/home/${var.ethereum_node.user}/ethereum_${local.ethereum_prepare_grafana_directories_filename}"
4+
}
5+
6+
resource "terraform_data" "ethereum_ec2_prepare_grafana_directories" {
7+
// 1st step is to copy the directory over to /home/${user} folder.
8+
// 2nd step is to execute the script
9+
10+
// Since it requires connection, it should depend on the ec2_instance
11+
12+
depends_on = [
13+
aws_instance.ethereum,
14+
terraform_data.ethereum_ec2_volume_preparation
15+
]
16+
17+
triggers_replace = {
18+
script_content = sha256(file("${local.bash_scripts_folder}/${local.ethereum_prepare_grafana_directories_filename}"))
19+
}
20+
21+
connection {
22+
type = "ssh"
23+
user = var.ethereum_node.user
24+
port = var.ethereum_node.ssh_port
25+
host = aws_instance.ethereum.public_ip
26+
private_key = file("${path.module}/ethereum.pem")
27+
}
28+
29+
// copy file over
30+
provisioner "file" {
31+
source = "${local.bash_scripts_folder}/${local.ethereum_prepare_grafana_directories_filename}"
32+
destination = local.destination_ethereum_prepare_grafana_directories_filename
33+
}
34+
35+
// make it executable and execute it
36+
provisioner "remote-exec" {
37+
inline = [
38+
"chmod +x ${local.destination_ethereum_prepare_grafana_directories_filename}",
39+
"sudo ${local.destination_ethereum_prepare_grafana_directories_filename} ${var.ethereum_node.user}",
40+
]
41+
}
42+
}

‎terraform/ethereum_prepare_prometheus_directories.tf‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
locals {
2+
ethereum_prepare_prometheus_directories_filename = "prepare_prometheus_directories.sh"
3+
destination_ethereum_prepare_prometheus_directories_filename = "/home/${var.ethereum_node.user}/ethereum_${local.ethereum_prepare_prometheus_directories_filename}"
4+
}
5+
6+
resource "terraform_data" "ethereum_ec2_prepare_prometheus_directories" {
7+
// 1st step is to copy the directory over to /home/${user} folder.
8+
// 2nd step is to execute the script
9+
10+
// Since it requires connection, it should depend on the ec2_instance
11+
12+
depends_on = [
13+
aws_instance.ethereum,
14+
terraform_data.ethereum_ec2_volume_preparation
15+
]
16+
17+
triggers_replace = {
18+
script_content = sha256(file("${local.bash_scripts_folder}/${local.ethereum_prepare_prometheus_directories_filename}"))
19+
}
20+
21+
connection {
22+
type = "ssh"
23+
user = var.ethereum_node.user
24+
port = var.ethereum_node.ssh_port
25+
host = aws_instance.ethereum.public_ip
26+
private_key = file("${path.module}/ethereum.pem")
27+
}
28+
29+
// copy file over
30+
provisioner "file" {
31+
source = "${local.bash_scripts_folder}/${local.ethereum_prepare_prometheus_directories_filename}"
32+
destination = local.destination_ethereum_prepare_prometheus_directories_filename
33+
}
34+
35+
// make it executable and execute it
36+
provisioner "remote-exec" {
37+
inline = [
38+
"chmod +x ${local.destination_ethereum_prepare_prometheus_directories_filename}",
39+
"sudo ${local.destination_ethereum_prepare_prometheus_directories_filename} ${var.ethereum_node.user}",
40+
]
41+
}
42+
}

‎terraform/locals.tf‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@ locals {
22
current_account_id = data.aws_caller_identity.current.account_id
33
current_user_arn = data.aws_caller_identity.current.arn
44

5-
templates_folder = "${path.module}/../templates"
5+
templates_folder = "${path.module}/../templates"
6+
bash_scripts_folder = "${path.module}/../bash"
67

78
cloudwatch_namespace = title("/${var.company}/${var.project}/${var.environment}")
89

‎terraform/production/ethereum_prepare_grafana_directories.tf‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../ethereum_prepare_grafana_directories.tf

0 commit comments

Comments
 (0)