Add orphaned test-VPC reaper and standardize dry-run across cleaners#2198
Open
sky333999 wants to merge 3 commits into
Open
Add orphaned test-VPC reaper and standardize dry-run across cleaners#2198sky333999 wants to merge 3 commits into
sky333999 wants to merge 3 commits into
Conversation
clean_eks: after the cluster pass, reap dedicated EFA test VPCs (tag:Name efa-test-vpc-*) left behind by failed terraform destroys. Tears the stack down in dependency order — VPC endpoints, NAT gateways, Elastic IPs, leftover ENIs, internet gateways, subnets, route tables, security groups (revoke-then-delete), then the VPC — behind three safety gates: not backing a live EKS cluster (prod + best-effort beta), no active EC2 instances (re-checked before subnet deletion), and an age gate from the newest NAT gateway/VPC endpoint. A cleaner:reaping tag lets a partially-torn-down VPC resume on a later run instead of being stranded once its age signals are gone. Shared dry-run: add tool/clean/clean_flags.go (clean.DryRun, RegisterCommonFlags, Skip) and gate every mutating call across all 12 cleaners. Add a -dry-run flag to each tool and a workflow_dispatch dry_run input (DRY_RUN env: scheduled runs stay live, manual defaults to dry-run). Normalize --tags=clean to build-flag position and move ecs/host region args to flag.Arg(0). Also fix pre-existing bugs surfaced while touching these files: - clean_ecs: inverted-sign expiration deleted clusters with active tasks - clean_security_group: break-in-select spun pagination to timeout - clean_ebs / clean_file_system: age no-op and stale-error under-deletion
sky333999
force-pushed
the
sky333999/efa-vpc-reaper
branch
from
6392598 to
9364aea
Compare
…waits, vet-clean Skip)
- vpcsInUseByClusters: treat ResourceNotFoundException (a cluster mid-deletion,
e.g. by terminateClusters just before) as not-in-use and continue, instead of
aborting the entire VPC reap pass; other errors still fail closed.
- waitVpcEndpointsGone / waitNatGatewaysGone: honor ctx cancellation during the
poll interval instead of an unconditional time.Sleep.
- clean.Skip: forward the format string so go vet recognizes it as a printf
wrapper and validates format/arg mismatches at every call site.
- Rename exported ClustersToClean to clustersToClean (package main, no external
consumers).
- Workflow: quote -dry-run="${DRY_RUN}" and correct the DRY_RUN comment.
- Note the single-Describe-page assumption on the per-VPC delete helpers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
clean_eks: after the cluster pass, reap dedicated EFA test VPCs (tag:Name efa-test-vpc-*) left behind by failed terraform destroys. Tears the stack down in dependency order — VPC endpoints, NAT gateways, Elastic IPs, leftover ENIs, internet gateways, subnets, route tables, security groups (revoke-then-delete), then the VPC — behind three safety gates: not backing a live EKS cluster (prod + best-effort beta), no active EC2 instances (re-checked before subnet deletion), and an age gate from the newest NAT gateway/VPC endpoint. A cleaner:reaping tag lets a partially-torn-down VPC resume on a later run instead of being stranded once its age signals are gone.
Shared dry-run: add tool/clean/clean_flags.go (clean.DryRun, RegisterCommonFlags, Skip) and gate every mutating call across all 12 cleaners. Add a -dry-run flag to each tool and a workflow_dispatch dry_run input (DRY_RUN env: scheduled runs stay live, manual defaults to dry-run). Normalize --tags=clean to build-flag position and move ecs/host region args to flag.Arg(0).
Also fix pre-existing bugs surfaced while touching these files:
License
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Tests
WIP
Requirements
Before commiting your code, please do the following steps.
make fmtandmake fmt-shmake lintIntegration Tests
To run integration tests against this PR, add the
ready for testinglabel.