Skip to content

Commit 21ef3b1

Browse files
ThomasK33ThomasK33
committed
docs: document S3 remote state backend and resource import
1 parent 884e6ee commit 21ef3b1

1 file changed

Lines changed: 191 additions & 3 deletions

File tree

‎terraform/README.md‎

Lines changed: 191 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,11 @@ This directory provisions a cost-optimized Amazon EKS sandbox cluster in region
1717
- Disk size: `20 GiB`
1818
- AMI type: `AL2023_x86_64_STANDARD`
1919
- EKS managed add-ons: `coredns`, `kube-proxy`, `vpc-cni`
20-
- Local Terraform state (no remote backend configured)
20+
- Remote Terraform state in AWS S3 with lockfile-based locking (no DynamoDB)
2121

2222
## Prerequisites
2323

24-
- Terraform `>= 1.5`
24+
- Terraform `>= 1.11`
2525
- AWS CLI v2 installed
2626
- AWS identity with permissions to create VPC, IAM, EKS, and EC2 resources in your target account
2727

@@ -71,14 +71,202 @@ If you prefer not to use `TF_VAR_aws_profile`, pass `-var="aws_profile=<your-ter
7171

7272
> Security note: do not commit `~/.aws/config`, `~/.aws/credentials`, or any copied credential values to git.
7373
74+
## Remote state backend (S3 + lockfile)
75+
76+
Terraform state is stored in an S3 bucket in `eu-central-1`. Locking uses Terraform's native S3 lockfile (`use_lockfile = true` in the backend block), so no DynamoDB lock table is required. The backend configuration committed in this repo only includes shared settings (`region`, `encrypt`, and `use_lockfile`); the bucket name and state key are supplied at init time via `-backend-config`.
77+
78+
### Bootstrap the S3 bucket (one-time)
79+
80+
Terraform cannot create its own backend bucket before backend initialization, so create and secure the bucket once with AWS CLI:
81+
82+
```bash
83+
REGION=eu-central-1
84+
BUCKET=<globally-unique-bucket-name>
85+
86+
# Create bucket
87+
aws s3api create-bucket \
88+
--bucket "$BUCKET" \
89+
--region "$REGION" \
90+
--create-bucket-configuration LocationConstraint="$REGION"
91+
92+
# Enable versioning (for state recovery)
93+
aws s3api put-bucket-versioning \
94+
--bucket "$BUCKET" \
95+
--versioning-configuration Status=Enabled
96+
97+
# Block all public access
98+
aws s3api put-public-access-block \
99+
--bucket "$BUCKET" \
100+
--public-access-block-configuration \
101+
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
102+
103+
# Default encryption (SSE-S3)
104+
aws s3api put-bucket-encryption \
105+
--bucket "$BUCKET" \
106+
--server-side-encryption-configuration '{
107+
"Rules": [{
108+
"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}
109+
}]
110+
}'
111+
```
112+
113+
### Required IAM permissions
114+
115+
The identity used for Terraform backend access needs S3 permissions for `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject` (lockfile lifecycle), and `s3:ListBucket` on the state bucket path.
116+
117+
### Authentication note
118+
119+
The S3 backend authenticates separately from the AWS provider configuration. If you use `TF_VAR_aws_profile` for provider auth, also set `AWS_PROFILE` or add `profile` to `backend.hcl` so `terraform init` can authenticate to S3.
120+
121+
### Migrate existing local state
122+
123+
If you already have a local `.tfstate` file, migrate it into S3 during init:
124+
125+
```bash
126+
terraform init -migrate-state -backend-config=backend.hcl
127+
```
128+
74129
## Usage
75130

131+
Create a local backend config file `backend.hcl` in this directory (it is
132+
git-ignored):
133+
134+
```hcl
135+
bucket = "<your-tfstate-bucket>"
136+
key = "terraform-ncp3/sandbox-eks/terraform.tfstate"
137+
138+
# Optional: set if you use an AWS CLI profile for backend access
139+
# profile = "terraform"
140+
```
141+
142+
Then initialize and apply:
143+
76144
```bash
77-
terraform init
145+
terraform init -backend-config=backend.hcl
78146
terraform plan
79147
terraform apply
80148
```
81149

150+
## Importing existing AWS resources
151+
152+
Use this flow when your remote state is empty but AWS infrastructure already exists.
153+
154+
- Do **not** run `terraform apply` until imports are complete and `terraform plan` is clean.
155+
- Ensure Terraform variables match the values used when the infrastructure was originally created.
156+
157+
<details>
158+
<summary>Discover resource IDs with AWS CLI</summary>
159+
160+
```bash
161+
REGION=eu-central-1
162+
CLUSTER=sandbox-eks
163+
164+
# VPC
165+
aws ec2 describe-vpcs --region "$REGION" \
166+
--filters "Name=tag:Name,Values=${CLUSTER}-vpc" \
167+
--query 'Vpcs[0].VpcId' --output text
168+
169+
# Internet Gateway
170+
aws ec2 describe-internet-gateways --region "$REGION" \
171+
--filters "Name=tag:Name,Values=${CLUSTER}-igw" \
172+
--query 'InternetGateways[0].InternetGatewayId' --output text
173+
174+
# Subnets (public + private)
175+
aws ec2 describe-subnets --region "$REGION" \
176+
--filters "Name=tag:Name,Values=${CLUSTER}-public-*" \
177+
--query 'Subnets[].{id:SubnetId,az:AvailabilityZone,name:Tags[?Key==`Name`]|[0].Value}' --output table
178+
179+
aws ec2 describe-subnets --region "$REGION" \
180+
--filters "Name=tag:Name,Values=${CLUSTER}-private-*" \
181+
--query 'Subnets[].{id:SubnetId,az:AvailabilityZone,name:Tags[?Key==`Name`]|[0].Value}' --output table
182+
183+
# EIP, NAT Gateway, Route Tables
184+
aws ec2 describe-addresses --region "$REGION" \
185+
--filters "Name=tag:Name,Values=${CLUSTER}-nat-eip" \
186+
--query 'Addresses[0].AllocationId' --output text
187+
188+
aws ec2 describe-nat-gateways --region "$REGION" \
189+
--filter "Name=tag:Name,Values=${CLUSTER}-nat" \
190+
--query 'NatGateways[0].NatGatewayId' --output text
191+
192+
aws ec2 describe-route-tables --region "$REGION" \
193+
--filters "Name=tag:Name,Values=${CLUSTER}-public-rt" \
194+
--query 'RouteTables[0].RouteTableId' --output text
195+
196+
aws ec2 describe-route-tables --region "$REGION" \
197+
--filters "Name=tag:Name,Values=${CLUSTER}-private-rt" \
198+
--query 'RouteTables[0].RouteTableId' --output text
199+
200+
# EKS cluster, node groups, addons
201+
aws eks describe-cluster --region "$REGION" --name "$CLUSTER" \
202+
--query 'cluster.name' --output text
203+
204+
aws eks list-nodegroups --region "$REGION" --cluster-name "$CLUSTER"
205+
aws eks list-addons --region "$REGION" --cluster-name "$CLUSTER"
206+
207+
# IAM roles
208+
aws iam get-role --role-name "${CLUSTER}-cluster-role" --query 'Role.RoleName' --output text
209+
aws iam get-role --role-name "${CLUSTER}-node-role" --query 'Role.RoleName' --output text
210+
```
211+
212+
</details>
213+
214+
<details>
215+
<summary>Import commands by Terraform file</summary>
216+
217+
### Networking (`vpc.tf`)
218+
219+
```bash
220+
terraform import aws_vpc.this <vpc-id>
221+
terraform import aws_internet_gateway.this <igw-id>
222+
terraform import 'aws_subnet.public[0]' <subnet-id>
223+
terraform import 'aws_subnet.public[1]' <subnet-id>
224+
terraform import 'aws_subnet.private[0]' <subnet-id>
225+
terraform import 'aws_subnet.private[1]' <subnet-id>
226+
terraform import aws_eip.nat <eipalloc-id>
227+
terraform import aws_nat_gateway.this <nat-id>
228+
terraform import aws_route_table.public <rtb-id>
229+
terraform import aws_route_table.private <rtb-id>
230+
terraform import 'aws_route_table_association.public[0]' <subnet-id>/<rtb-id>
231+
terraform import 'aws_route_table_association.public[1]' <subnet-id>/<rtb-id>
232+
terraform import 'aws_route_table_association.private[0]' <subnet-id>/<rtb-id>
233+
terraform import 'aws_route_table_association.private[1]' <subnet-id>/<rtb-id>
234+
```
235+
236+
### IAM (`iam.tf`)
237+
238+
```bash
239+
terraform import aws_iam_role.eks_cluster sandbox-eks-cluster-role
240+
terraform import aws_iam_role.node_group sandbox-eks-node-role
241+
terraform import aws_iam_role_policy_attachment.eks_cluster_policy \
242+
'sandbox-eks-cluster-role/arn:aws:iam::aws:policy/AmazonEKSClusterPolicy'
243+
terraform import aws_iam_role_policy_attachment.node_group_worker_policy \
244+
'sandbox-eks-node-role/arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy'
245+
terraform import aws_iam_role_policy_attachment.node_group_cni_policy \
246+
'sandbox-eks-node-role/arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy'
247+
terraform import aws_iam_role_policy_attachment.node_group_ecr_readonly \
248+
'sandbox-eks-node-role/arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly'
249+
```
250+
251+
### EKS (`eks.tf`)
252+
253+
```bash
254+
terraform import aws_eks_cluster.this sandbox-eks
255+
terraform import aws_eks_node_group.this sandbox-eks:sandbox-eks-nodes
256+
terraform import aws_eks_addon.coredns sandbox-eks:coredns
257+
terraform import aws_eks_addon.kube_proxy sandbox-eks:kube-proxy
258+
terraform import aws_eks_addon.vpc_cni sandbox-eks:vpc-cni
259+
```
260+
261+
</details>
262+
263+
After importing, run `terraform plan` and confirm the result is ideally a no-op.
264+
265+
Common post-import issues:
266+
267+
- Subnet index/AZ mismatch can cause Terraform to propose subnet recreation; fix by swapping imported subnet indices (`public[0]`/`public[1]`, `private[0]`/`private[1]`) to match Terraform's AZ ordering.
268+
- EKS add-on configuration drift may still appear depending on AWS-managed defaults; review and reconcile intentionally.
269+
82270
## Configure kubectl
83271

84272
After `terraform apply`, run the command from the Terraform output:

0 commit comments

Comments
 (0)