Skip to content

Commit 3138da8

Browse files
pmatsinopoulospmatsinopoulos
authored
L2 op stack chains (#17)
* First phase to build the l2 op stack: Next install reth on l2 op stack node * Added 1TB to the base node * SSM Agent logs * Base production chain is 'base-mainnet' * L2 node is ready to start syncing * Enabled prometheus and grafana and metric-exporter for base/L2 * L2 node added nginx * Load Balancer for l2 base node * [doc] How to make new storage available * Load balancers for Base / L2 nodes * Name of the l1 instance to end with the chain name * ebs volume iops and throughput as variables * NGINX port only available from within the VPC * web socket port exposed from containers * Route 53 entries to allow for internal names to be used
1 parent 7e3c1b0 commit 3138da8

70 files changed

Lines changed: 1920 additions & 38 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/HOW_TO_USE_THE_NEW_ALLOCATED_EBS_STORAGE.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
When I allocate more storage space for my EBS volume, I then need to follow these instructions
2+
to make this space available to the `/data` folder.
3+
4+
1. Verify the block device sees the new size
5+
6+
Run:
7+
8+
```
9+
sudo lsblk -o NAME,SIZE -d /dev/nvme1n1
10+
```
11+
12+
2. Grow the XFS filesystem in place
13+
14+
Since XFS supports online growth, you just need:
15+
16+
```
17+
sudo xfs_growfs /data
18+
```
19+
20+
3. Confirm
21+
22+
```
23+
df -hT /data
24+
```
25+
26+
You should now see the new total size

‎terraform/base_ec2_instance.tf‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
module "base_ec2_instance" {
2+
source = "./modules/l2_op_stack_chains"
3+
4+
company = var.company
5+
current_account_id = local.current_account_id
6+
project = var.project
7+
parent_project = var.parent_project
8+
environment = var.environment
9+
cloudwatch_namespace = local.cloudwatch_namespace
10+
bash_scripts_folder = "${path.module}/../bash"
11+
install_docker_script_name = "install_docker.sh"
12+
configure_ssm_agent_script_name = "configure_ssm_agent_logs.sh"
13+
region = var.region
14+
load_balancer = var.load_balancer
15+
16+
node = {
17+
alarm_memory_threshold_percentage = var.base_node.alarm_memory_threshold_percentage # 80
18+
alarm_disk_used_threshold_percentage = var.base_node.alarm_disk_used_threshold_percentage # 90
19+
ami_id = var.base_node.ami_id # "ami-0122205e4fe2524d5" # ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-arm64-server-20250821
20+
associate_public_ip_address = var.base_node.associate_public_ip_address # true
21+
availability_zone = var.base_node.availability_zone # "a"
22+
chain = var.base_node.chain
23+
# checkpoint_sync_url = var.base_node.checkpoint_sync_url
24+
ebs_optimized = var.base_node.ebs_optimized
25+
ebs_volume = var.base_node.ebs_volume
26+
instance_type = var.base_node.instance_type
27+
logs = {
28+
retention_days = var.base_node.logs.retention_days
29+
}
30+
key_pair_pem_file_path = "${path.module}/ethereum.pem"
31+
monitoring = var.base_node.monitoring
32+
root_block_device = var.base_node.root_block_device
33+
security_group_ids = [
34+
local.vpc_default_security_group_id,
35+
aws_security_group.ethereum_ssh.id,
36+
aws_security_group.ethereum_nginx.id,
37+
aws_security_group.ethereum_public_p2p.id,
38+
aws_security_group.op_node_public_p2p.id,
39+
]
40+
ssh_port = var.base_node.ssh_port
41+
subnet_id = local.public_subnets_per_availability_zone[var.base_node.availability_zone].id
42+
user = var.base_node.user
43+
user_data = templatefile("${local.templates_folder}/ethereum_user_data_cloud_init.sh.tftpl", {
44+
ssh_port = var.base_node.ssh_port
45+
user = var.base_node.user
46+
})
47+
}
48+
49+
node_name = var.base_node.chain
50+
node_iam_instance_profile_name = aws_iam_instance_profile.ethereum_node.name
51+
node_key_pair_name = aws_key_pair.ethereum.key_name
52+
node_subnet_id = local.public_subnets_per_availability_zone[var.base_node.availability_zone].id
53+
node_user_data = templatefile("${local.templates_folder}/ethereum_user_data_cloud_init.sh.tftpl", {
54+
ssh_port = var.base_node.ssh_port
55+
user = var.base_node.user
56+
})
57+
58+
ssm_agent_log_file_configuration = "${path.module}/../ssm-agent-seelog.xml"
59+
60+
gf_security_admin_user = var.gf_security_admin_user
61+
gf_security_admin_password = var.gf_security_admin_password
62+
prometheus_admin_password = var.prometheus_admin_password
63+
prometheus_admin_password_bcrypted = var.prometheus_admin_password_bcrypted
64+
65+
sequencer_http = var.base_node.sequencer_http
66+
beacon_node_http_endpoint = "http://${aws_instance.ethereum.private_ip}:5052"
67+
op_reth_version = var.base_node.op_reth_version
68+
op_node_version = var.base_node.op_node_version
69+
op_node_network = var.base_node.op_node_network
70+
l1_rpc = "http://${aws_instance.ethereum.private_ip}:8545"
71+
72+
nginx = var.nginx
73+
74+
vpc_id = data.aws_vpc.main.id
75+
76+
public_subnet_ids = [
77+
for subnet in local.public_subnets : subnet.id
78+
]
79+
80+
load_balancer_security_group_ids = [
81+
local.vpc_default_security_group_id,
82+
data.aws_security_group.http_80.id,
83+
data.aws_security_group.https_443.id
84+
]
85+
86+
load_balancer_certificate_arn = data.aws_acm_certificate.wildcard_talentprotocol_com.arn
87+
88+
private_zone_id = data.aws_route53_zone.talent_protocol_internal.zone_id
89+
90+
private_zone_name = data.aws_route53_zone.talent_protocol_internal.name
91+
}
92+
93+
output "base_ec2_instance_ssh_connect" {
94+
value = module.base_ec2_instance.ec2_instance_ssh_connect
95+
description = "SSH command to connect to the base EC2 instance"
96+
}

‎terraform/bash/configure_ssm_agent_logs.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
3+
set -e # e: exit if any command has a non-zero exit status
4+
set -x # x: all executed commands are printed to the terminal
5+
set -u # u: all references to variables that have not been previously defined cause an error
6+
7+
USER=$1
8+
9+
sudo mkdir -p /etc/amazon/ssm
10+
sudo mv /home/${USER}/seelog.xml /etc/amazon/ssm/seelog.xml
11+
sudo chown root:root /etc/amazon/ssm/seelog.xml
12+
sudo chown -R root:adm /var/log/amazon
13+
sudo chmod u+rwx,g+rx,g-w,o+rx,o-w /var/log/amazon
14+
sudo chmod u+rwx,g+rx,g-w,o+rx,o-w /var/log/amazon/ssm
15+
sudo systemctl restart snap.amazon-ssm-agent.amazon-ssm-agent.service

‎terraform/bash/extend_ebs_volume.sh‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
#!/bin/bash
2+
3+
set -e # e: exit if any command has a non-zero exit status
4+
set -x # x: all executed commands are printed to the terminal
5+
set -u # u: all references to variables that have not been previously defined cause an error
6+
7+
# Verify the block device sees the new size
8+
sudo lsblk -o NAME,SIZE -d /dev/nvme1n1
9+
10+
# Grow the XFS filesystem in place
11+
# Since XFS supports online growth, you just need:
12+
#
13+
sudo xfs_growfs /data
14+
15+
# Confirm
16+
df -hT /data

terraform/install_docker.sh renamed to terraform/bash/install_docker.sh

File renamed without changes.

‎terraform/data.tf‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,8 @@ data "aws_iam_policy" "ssm_managed_instance_core" {
5252
data "aws_s3_bucket" "ssm_associations_output" {
5353
bucket = "${var.company}-${var.core_infrastructure_project}-ssm-associations-output"
5454
}
55+
56+
data "aws_route53_zone" "talent_protocol_internal" {
57+
name = "talentprotocol.internal"
58+
private_zone = true
59+
}

‎terraform/ethereum_ec2_cloudwatch_agent_configuration_update.tf‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ resource "local_file" "ethereum_ec2_install_cloudwatch_agent" {
55

66
resource "terraform_data" "ethereum_ec2_cloudwatch_agent_configuration_update" {
77
depends_on = [
8+
aws_instance.ethereum,
9+
aws_eip.ethereum,
810
terraform_data.ethereum_ec2_install_cloudwatch_agent,
911
local_file.ethereum_ec2_install_cloudwatch_agent
1012
]

‎terraform/ethereum_ec2_install_cloudwatch_agent.tf‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
resource "terraform_data" "ethereum_ec2_install_cloudwatch_agent" {
22
depends_on = [
33
aws_instance.ethereum,
4-
terraform_data.ethereum_ec2_install_docker
4+
aws_eip.ethereum
55
]
66

77
connection {

‎terraform/ethereum_ec2_install_docker.tf‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
1+
locals {
2+
install_docker_script_name = "install_docker.sh"
3+
}
4+
15
resource "terraform_data" "ethereum_ec2_install_docker" {
2-
depends_on = [terraform_data.ethereum_ec2_volume_preparation]
6+
depends_on = [
7+
aws_instance.ethereum,
8+
aws_eip.ethereum
9+
]
310

411
triggers_replace = {
512
volume_attachment = aws_volume_attachment.ethereum.id
@@ -14,15 +21,15 @@ resource "terraform_data" "ethereum_ec2_install_docker" {
1421
}
1522

1623
provisioner "file" {
17-
source = "${path.module}/../install_docker.sh"
18-
destination = "/home/${var.ethereum_node.user}/install_docker.sh"
24+
source = "${local.bash_scripts_folder}/${local.install_docker_script_name}"
25+
destination = "/home/${var.ethereum_node.user}/${local.install_docker_script_name}"
1926
}
2027

2128
provisioner "remote-exec" {
2229
inline = concat(
2330
[
24-
"sudo chmod u+x /home/${var.ethereum_node.user}/install_docker.sh",
25-
"sudo /home/${var.ethereum_node.user}/install_docker.sh ${var.ethereum_node.user}",
31+
"sudo chmod u+x /home/${var.ethereum_node.user}/${local.install_docker_script_name}",
32+
"sudo /home/${var.ethereum_node.user}/${local.install_docker_script_name} ${var.ethereum_node.user}",
2633
]
2734
)
2835
}

‎terraform/ethereum_ec2_install_lz4.tf‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
resource "terraform_data" "install_lz4" {
2-
depends_on = [aws_instance.ethereum]
2+
depends_on = [
3+
aws_instance.ethereum,
4+
aws_eip.ethereum
5+
]
36

47
connection {
58
type = "ssh"

0 commit comments

Comments
 (0)