Skip to content

Commit 5715649

Browse files
committed
feat: Add gcloud community module
This commit introduces a new community module to execute gcloud commands during Cluster Toolkit deployments. The module allows specifying create and delete commands for resources, providing a way to manage resources not yet available in Terraform or the toolkit. This is particularly useful for rapid prototyping and leveraging new GCP features quickly. Includes an example blueprint demonstrating network, subnet, and VM creation and deletion.
1 parent 0a4d07b commit 5715649

9 files changed

Lines changed: 274 additions & 0 deletions

File tree

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# This blueprint demonstrates the use of the gcloud community module to create a network, subnet, and VM.
16+
17+
blueprint_name: gcloud-example
18+
19+
vars:
20+
project_id: # Set Your Project ID
21+
region: us-central1
22+
zone: us-central1-a
23+
deployment_name: gcloud-test
24+
25+
deployment_groups:
26+
- group: primary
27+
modules:
28+
- id: gcloud_commands
29+
source: community/modules/scripts/gcloud
30+
settings:
31+
commands:
32+
- create: "gcloud compute networks create $(vars.deployment_name)-net --project=$(vars.project_id) --subnet-mode=custom"
33+
delete: "gcloud compute networks delete $(vars.deployment_name)-net --project=$(vars.project_id) --quiet"
34+
- create: "gcloud compute networks subnets create $(vars.deployment_name)-subnet --project=$(vars.project_id) --network=$(vars.deployment_name)-net --range=10.10.0.0/24 --region=$(vars.region)"
35+
delete: "gcloud compute networks subnets delete $(vars.deployment_name)-subnet --project=$(vars.project_id) --region=$(vars.region) --quiet"
36+
- create: >-
37+
gcloud compute instances create $(vars.deployment_name)-vm
38+
--project=$(vars.project_id)
39+
--zone=$(vars.zone)
40+
--network=$(vars.deployment_name)-net
41+
--subnet=$(vars.deployment_name)-subnet
42+
--machine-type=e2-micro
43+
--image-family=debian-11
44+
--image-project=debian-cloud
45+
delete: "gcloud compute instances delete $(vars.deployment_name)-vm --project=$(vars.project_id) --zone=$(vars.zone) --quiet"
Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# gcloud Module
2+
3+
This module allows you to run a series of gcloud commands as part of a Cluster Toolkit deployment.
4+
5+
## Usage
6+
7+
```yaml
8+
- id: my_gcloud_steps
9+
source: community/modules/gcloud
10+
settings:
11+
commands:
12+
- gcloud compute networks create my-network --subnet-mode=custom
13+
- gcloud compute networks subnets create my-subnet --network=my-network --range=10.0.0.0/24 --region=us-central1
14+
- gcloud compute instances create my-vm --zone=us-central1-a --network=my-network --subnet=my-subnet --machine-type=e2-medium
15+
## Dependency Management
16+
17+
This module uses `local-exec` provisioners to run `gcloud` commands. As such, it does not expose any outputs that other Terraform modules can consume to establish dependencies.
18+
19+
To ensure that resources managed by this module are fully provisioned before other modules in subsequent deployment groups run, you **must** place this `gcloud` module in a deployment group that is ordered *before* any groups that depend on the resources it creates.
20+
21+
For example:
22+
23+
```yaml
24+
deployment_groups:
25+
- group: gcloud_setup
26+
modules:
27+
- id: my_gcloud_commands
28+
source: community/modules/scripts/gcloud
29+
settings:
30+
# ... gcloud commands to create a network and subnet
31+
32+
- group: vm_deployment
33+
modules:
34+
- id: my_vms
35+
source: ./modules/compute/vm
36+
settings:
37+
# This module can assume the network and subnet from the gcloud_setup group exist
38+
network: my-network
39+
subnet: my-subnet
40+
# ... other vm settings
41+
```
42+
43+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
44+
## Requirements
45+
46+
| Name | Version |
47+
|------|---------|
48+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13 |
49+
| <a name="requirement_null"></a> [null](#requirement\_null) | >= 3.0 |
50+
51+
## Providers
52+
53+
| Name | Version |
54+
|------|---------|
55+
| <a name="provider_null"></a> [null](#provider\_null) | >= 3.0 |
56+
57+
## Modules
58+
59+
No modules.
60+
61+
## Resources
62+
63+
| Name | Type |
64+
|------|------|
65+
| [null_resource.gcloud_commands](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) | resource |
66+
67+
## Inputs
68+
69+
| Name | Description | Type | Default | Required |
70+
|------|-------------|------|---------|:--------:|
71+
| <a name="input_commands"></a> [commands](#input\_commands) | A list of gcloud command pairs for creation and destruction. | <pre>list(object({<br/> create = string<br/> delete = string<br/> }))</pre> | `[]` | no |
72+
| <a name="input_module_instance_id"></a> [module\_instance\_id](#input\_module\_instance\_id) | The unique ID of this module instance in the blueprint. This is automatically populated by gcluster. | `string` | n/a | yes |
73+
74+
## Outputs
75+
76+
No outputs.
77+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
locals {
16+
create_script_content = <<EOT
17+
#!/bin/bash
18+
set -e -x
19+
echo "Executing gcloud CREATE commands for ${var.module_instance_id}..."
20+
%{for cmd_pair in var.commands~}
21+
${cmd_pair.create}
22+
%{endfor~}
23+
echo "All gcloud CREATE commands executed successfully for ${var.module_instance_id}."
24+
EOT
25+
26+
destroy_script_content = <<EOT
27+
#!/bin/bash
28+
set -x # We don't use -e here to allow other cleanup commands to run if one fails
29+
echo "Executing gcloud DELETE commands for ${var.module_instance_id}..."
30+
%{for cmd_pair in reverse(var.commands)~}
31+
${cmd_pair.delete} || echo "Delete command failed: ${cmd_pair.delete}, continuing..."
32+
%{endfor~}
33+
echo "All gcloud DELETE commands attempted for ${var.module_instance_id}."
34+
EOT
35+
}
36+
37+
resource "null_resource" "gcloud_commands" {
38+
triggers = {
39+
commands_hash = sha256(jsonencode(var.commands))
40+
module_instance_id = var.module_instance_id
41+
create_script_content = local.create_script_content
42+
destroy_script_content = local.destroy_script_content
43+
}
44+
45+
provisioner "local-exec" {
46+
command = "/bin/bash <<EOT\n${self.triggers.create_script_content}\nEOT"
47+
}
48+
49+
provisioner "local-exec" {
50+
when = destroy
51+
command = "/bin/bash <<EOT\n${self.triggers.destroy_script_content}\nEOT"
52+
}
53+
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
---
16+
spec:
17+
requirements:
18+
services: []
19+
ghpc:
20+
inject_module_id: module_instance_id
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# No outputs for this module
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
variable "commands" {
16+
description = "A list of gcloud command pairs for creation and destruction."
17+
type = list(object({
18+
create = string
19+
delete = string
20+
}))
21+
default = []
22+
}
23+
24+
variable "module_instance_id" {
25+
26+
description = "The unique ID of this module instance in the blueprint. This is automatically populated by gcluster."
27+
28+
type = string
29+
30+
}
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Copyright 2025 "Google LLC"
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
terraform {
16+
required_providers {
17+
null = {
18+
source = "hashicorp/null"
19+
version = ">= 3.0"
20+
}
21+
}
22+
required_version = ">= 0.13"
23+
}

examples/README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ md_toc github examples/README.md | sed -e "s/\s-\s/ * /"
6666
* [gke-g4](#gke-g4-) ![core-badge]
6767
* [netapp-volumes.yaml](#netapp-volumesyaml--) ![core-badge]
6868
* [gke-tpu-7x](#gke-tpu-7x-) ![core-badge]
69+
* [gcloud-example.yaml](#gcloud-exampleyaml--) ![community-badge] ![experimental-badge]
6970
* [Blueprint Schema](#blueprint-schema)
7071
* [Writing an HPC Blueprint](#writing-an-hpc-blueprint)
7172
* [Blueprint Boilerplate](#blueprint-boilerplate)
@@ -1705,6 +1706,14 @@ This example shows how TPU 7x cluster can be created and be used to run a job th
17051706

17061707
[gke-tpu-7x]: ../examples/gke-tpu-7x
17071708

1709+
### [gcloud-example.yaml] ![community-badge] ![experimental-badge]
1710+
1711+
This blueprint demonstrates how to use the `gcloud` community module to run
1712+
arbitrary `gcloud` commands during deployment and destroy. It shows an example
1713+
of creating and deleting a network, subnet, and VM instance.
1714+
1715+
[gcloud-example.yaml]: ../community/examples/gcloud-example.yaml
1716+
17081717
## Blueprint Schema
17091718

17101719
Similar documentation can be found on

modules/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@ Pub/Sub subscription. Primarily used for [FSI - MonteCarlo Tutorial][fsi-monteca
236236
software build using [Spack](https://github.com/spack/spack).
237237
* **[wait-for-startup]** ![community-badge] ![experimental-badge] : Waits for
238238
successful completion of a startup script on a compute VM.
239+
* **[gcloud]** ![community-badge] ![experimental-badge] : Executes arbitrary `gcloud` commands with create/destroy lifecycle.
239240

240241
[startup-script]: scripts/startup-script/README.md
241242
[windows-startup-script]: ../community/modules/scripts/windows-startup-script/README.md
@@ -246,6 +247,7 @@ Pub/Sub subscription. Primarily used for [FSI - MonteCarlo Tutorial][fsi-monteca
246247
[spack-setup]: ../community/modules/scripts/spack-setup/README.md
247248
[spack-execute]: ../community/modules/scripts/spack-execute/README.md
248249
[wait-for-startup]: ../community/modules/scripts/wait-for-startup/README.md
250+
[gcloud]: ../community/modules/scripts/gcloud/README.md
249251

250252
## Module Fields
251253

0 commit comments

Comments
 (0)