Skip to content

Commit d2446a6

Browse files
arampricecursoragentCopilotneddp
authored
Add job to ensure expected test networks are created (#591)
* Add job to ensure expected GCP integration network is created Why --- The bats and test-stemcells-ipv4 jobs assume a GCP subnetwork named stemcell-builder-integration-<subnet_int> exists in the bosh-concourse VPC, with a /24 at 10.100.<subnet_int>.0/24, private Google access, and IPV4_ONLY stack type. They also require a matching ingress firewall rule (all-protocol, source CIDR → tags test-stemcells-bats/bat) so that compilation VMs and BAT deployment VMs can reach the BOSH director's NATS server. Until now both resources had to be created and maintained out of band; their absence caused consistent compilation-VM agent timeouts (builds 466–475). What ---- * ci/tasks/gcp/ensure-integration-network.sh - Authenticates via GCP_JSON_KEY service account. - Derives SUBNET_NAME and SUBNET_CIDR from SUBNET_INT. - Captures stderr via mktemp temp file (cleaned up by trap on EXIT) so that gcloud failures are classified: "was not found" → create the resource; anything else → print the error and exit non-zero. This prevents auth/permission/transient API errors from being silently misinterpreted as "resource missing". - Subnetwork: single gcloud describe call captures exit code (for existence) and attributes (for drift detection). Validates network, ipCidrRange, privateIpGoogleAccess, and stackType; exits non-zero with a clear diff on any mismatch. - Firewall rule: same stderr-capture pattern. Validates network, direction, allowed[0].IPProtocol (must be "all"), sourceRanges[0], and disabled (must be False) in one describe call. Validates targetTags in a second describe call, sorting both sides before comparison to be order-insensitive. Both 'test-stemcells-bats' and 'bat' tags are required, mirroring the existing stemcell-builder-integration-22 rule. * ci/tasks/gcp/ensure-integration-network.yml - Concourse task definition. All params (GCP_JSON_KEY, GCP_PROJECT_ID, GCP_REGION, GCP_NETWORK_NAME, SUBNET_INT) are required; no defaults, values are provided explicitly by the pipeline. * ci/pipelines/builder.yml - New infrastructure group containing the new job. - New job ensure-integration-network: * serial: true, manual trigger only. * Gets bosh-stemcells-ci and bosh-integration-image, then runs the task with GCP_REGION=europe-north2 and GCP_NETWORK_NAME=bosh-concourse passed explicitly. * No passed: constraint on existing jobs; run on demand when the subnet/firewall needs to be created or reconciled. Verification ------------ * ytt -f ci/pipelines/builder.yml -f ci/pipelines/vars.yml renders successfully. * fly validate-pipeline -c <rendered> reports "looks good". Co-authored-by: Cursor <cursoragent@cursor.com> * Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Harden firewall existence check and validation in ensure-integration-network --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Ned Petrov <nedd.petrov@gmail.com>
1 parent cb2082a commit d2446a6

3 files changed

Lines changed: 176 additions & 1 deletion

File tree

‎ci/pipelines/builder.yml‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,9 @@ groups:
4444
- name: docker
4545
jobs:
4646
- build-os-image-stemcell-builder
47+
- name: infrastructure
48+
jobs:
49+
- ensure-integration-network
4750

4851
#@yaml/text-templated-strings
4952
jobs:
@@ -89,6 +92,25 @@ jobs:
8992
get_params:
9093
skip_download: true
9194

95+
#! Manually triggered job that idempotently ensures the GCP subnetwork and
96+
#! firewall rule consumed by deploy-director / cleanup-bats-vms / prepare-bats
97+
#! in the test-stemcells-ipv4 and bats jobs below exist. GCP is the source of
98+
#! truth — no state file is required.
99+
- name: ensure-integration-network
100+
serial: true
101+
plan:
102+
- get: bosh-stemcells-ci
103+
- get: bosh-integration-image
104+
- task: ensure-integration-network
105+
file: bosh-stemcells-ci/ci/tasks/gcp/ensure-integration-network.yml
106+
image: bosh-integration-image
107+
params:
108+
GCP_JSON_KEY: ((gcp_json_key))
109+
GCP_PROJECT_ID: ((gcp_project_id))
110+
GCP_REGION: europe-north2
111+
GCP_NETWORK_NAME: bosh-concourse
112+
SUBNET_INT: (@= data.values.stemcell_details.subnet_int @)
113+
92114
- name: process-high-critical-cves
93115
serial_groups: [log-cves]
94116
plan:
@@ -885,7 +907,6 @@ resource_types:
885907
type: registry-image
886908
source:
887909
repository: frodenas/gcs-resource
888-
889910
#@yaml/text-templated-strings
890911
resources:
891912
- name: daily

‎ci/tasks/gcp/ensure-integration-network.sh‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
#!/usr/bin/env bash
2+
set -eu -o pipefail
3+
4+
: "${GCP_JSON_KEY:?}"
5+
: "${GCP_PROJECT_ID:?}"
6+
: "${GCP_REGION:?}"
7+
: "${GCP_NETWORK_NAME:?}"
8+
: "${SUBNET_INT:?}"
9+
10+
echo "${GCP_JSON_KEY}" | gcloud auth activate-service-account --key-file - --project "${GCP_PROJECT_ID}"
11+
12+
SUBNET_NAME="stemcell-builder-integration-${SUBNET_INT}"
13+
SUBNET_CIDR="10.100.${SUBNET_INT}.0/24"
14+
15+
# 'bat' => BATS created VM tag
16+
# 'test-stemcells-bats' => director, and compilation VM tag
17+
FIREWALL_TAGS="bat,test-stemcells-bats"
18+
19+
gcloud_stderr="$(mktemp)"
20+
trap 'rm -f "${gcloud_stderr}"' EXIT
21+
22+
echo "Checking for subnet '${SUBNET_NAME}' in region '${GCP_REGION}'..."
23+
existing_subnet_name="$(gcloud compute networks subnets list \
24+
--regions="${GCP_REGION}" \
25+
--project="${GCP_PROJECT_ID}" \
26+
--filter="name=('${SUBNET_NAME}')" \
27+
--format='value(name)' \
28+
2>"${gcloud_stderr}")" && subnet_lookup_ok=true || subnet_lookup_ok=false
29+
30+
if ${subnet_lookup_ok}; then
31+
if [[ -n "${existing_subnet_name}" ]]; then
32+
current_subnet="$(gcloud compute networks subnets describe "${SUBNET_NAME}" \
33+
--region="${GCP_REGION}" \
34+
--project="${GCP_PROJECT_ID}" \
35+
--format='csv[no-heading](network.basename(),ipCidrRange,privateIpGoogleAccess,stackType)' \
36+
2>"${gcloud_stderr}")"
37+
expected_subnet="${GCP_NETWORK_NAME},${SUBNET_CIDR},True,IPV4_ONLY"
38+
if [[ "${current_subnet}" != "${expected_subnet}" ]]; then
39+
echo "ERROR: Subnet '${SUBNET_NAME}' exists but is misconfigured."
40+
echo " Expected: ${expected_subnet}"
41+
echo " Actual: ${current_subnet}"
42+
exit 1
43+
fi
44+
echo "Subnet '${SUBNET_NAME}' already exists and matches expected configuration."
45+
else
46+
echo "Creating subnet '${SUBNET_NAME}'..."
47+
gcloud compute networks subnets create "${SUBNET_NAME}" \
48+
--network="${GCP_NETWORK_NAME}" \
49+
--region="${GCP_REGION}" \
50+
--range="${SUBNET_CIDR}" \
51+
--enable-private-ip-google-access \
52+
--stack-type=IPV4_ONLY \
53+
--project="${GCP_PROJECT_ID}"
54+
echo "Subnet '${SUBNET_NAME}' created."
55+
fi
56+
else
57+
echo "ERROR: gcloud subnet lookup failed for subnet '${SUBNET_NAME}':"
58+
cat "${gcloud_stderr}" >&2
59+
exit 1
60+
fi
61+
62+
echo "Checking for firewall rule '${SUBNET_NAME}'..."
63+
existing_fw_name="$(gcloud compute firewall-rules list \
64+
--project="${GCP_PROJECT_ID}" \
65+
--filter="name=('${SUBNET_NAME}')" \
66+
--format='value(name)' \
67+
2>"${gcloud_stderr}")" && fw_lookup_ok=true || fw_lookup_ok=false
68+
69+
if ${fw_lookup_ok}; then
70+
if [[ -n "${existing_fw_name}" ]]; then
71+
current_fw_json="$(gcloud compute firewall-rules describe "${SUBNET_NAME}" \
72+
--project="${GCP_PROJECT_ID}" \
73+
--format=json \
74+
2>"${gcloud_stderr}")"
75+
76+
# Validate network, direction, disabled
77+
actual_network="$(echo "${current_fw_json}" | jq -r '.network | split("/") | last')"
78+
actual_direction="$(echo "${current_fw_json}" | jq -r '.direction')"
79+
actual_disabled="$(echo "${current_fw_json}" | jq -r '.disabled')"
80+
81+
if [[ "${actual_network}" != "${GCP_NETWORK_NAME}" ]] || \
82+
[[ "${actual_direction}" != "INGRESS" ]] || \
83+
[[ "${actual_disabled}" != "false" ]]; then
84+
echo "ERROR: Firewall rule '${SUBNET_NAME}' exists but is misconfigured."
85+
echo " Expected network=${GCP_NETWORK_NAME}, direction=INGRESS, disabled=false"
86+
echo " Actual network=${actual_network}, direction=${actual_direction}, disabled=${actual_disabled}"
87+
exit 1
88+
fi
89+
90+
# Validate allowed (should be exactly [{IPProtocol: "all"}])
91+
actual_allowed="$(echo "${current_fw_json}" | jq -c '[.allowed[] | {protocol: .IPProtocol, ports: (.ports // [])}] | sort_by(.protocol)')"
92+
expected_allowed='[{"protocol":"all","ports":[]}]'
93+
if [[ "${actual_allowed}" != "${expected_allowed}" ]]; then
94+
echo "ERROR: Firewall rule '${SUBNET_NAME}' has wrong allowed configuration."
95+
echo " Expected: ${expected_allowed}"
96+
echo " Actual: ${actual_allowed}"
97+
exit 1
98+
fi
99+
100+
# Validate sourceRanges (should be exactly the subnet CIDR)
101+
actual_ranges="$(echo "${current_fw_json}" | jq -c '(.sourceRanges // []) | sort')"
102+
expected_ranges="$(printf '["%s"]' "${SUBNET_CIDR}")"
103+
if [[ "${actual_ranges}" != "${expected_ranges}" ]]; then
104+
echo "ERROR: Firewall rule '${SUBNET_NAME}' has wrong source ranges."
105+
echo " Expected: ${expected_ranges}"
106+
echo " Actual: ${actual_ranges}"
107+
exit 1
108+
fi
109+
110+
# Validate targetTags (order-insensitive)
111+
actual_tags="$(echo "${current_fw_json}" | jq -c '(.targetTags // []) | sort')"
112+
expected_tags="$(printf '%s\n' ${FIREWALL_TAGS//,/ } | jq -R . | jq -sc 'sort')"
113+
if [[ "${actual_tags}" != "${expected_tags}" ]]; then
114+
echo "ERROR: Firewall rule '${SUBNET_NAME}' has wrong target tags."
115+
echo " Expected: ${expected_tags}"
116+
echo " Actual: ${actual_tags}"
117+
exit 1
118+
fi
119+
120+
echo "Firewall rule '${SUBNET_NAME}' already exists and matches expected configuration."
121+
else
122+
echo "Creating firewall rule '${SUBNET_NAME}'..."
123+
gcloud compute firewall-rules create "${SUBNET_NAME}" \
124+
--network="${GCP_NETWORK_NAME}" \
125+
--project="${GCP_PROJECT_ID}" \
126+
--direction=INGRESS \
127+
--priority=1000 \
128+
--allow=all \
129+
--source-ranges="${SUBNET_CIDR}" \
130+
--target-tags="${FIREWALL_TAGS}"
131+
echo "Firewall rule '${SUBNET_NAME}' created."
132+
fi
133+
else
134+
echo "ERROR: gcloud firewall-rules lookup failed for '${SUBNET_NAME}':"
135+
cat "${gcloud_stderr}" >&2
136+
exit 1
137+
fi
138+
139+
echo "Integration network '${SUBNET_NAME}' is ready."

‎ci/tasks/gcp/ensure-integration-network.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
platform: linux
3+
4+
inputs:
5+
- name: bosh-stemcells-ci
6+
7+
params:
8+
GCP_JSON_KEY:
9+
GCP_PROJECT_ID:
10+
GCP_REGION:
11+
GCP_NETWORK_NAME:
12+
SUBNET_INT:
13+
14+
run:
15+
path: bosh-stemcells-ci/ci/tasks/gcp/ensure-integration-network.sh

0 commit comments

Comments
 (0)