Skip to content

Commit fdd04fb

Browse files
samskillmansamskillman
authored
Merge pull request GoogleCloudPlatform#4538 from wiktorn/fix_inconsistent_plan
Fix inconsistent plan on Slurm cluster reconfigure
2 parents 86c66ca + 9c1caf4 commit fdd04fb

12 files changed

Lines changed: 82 additions & 66 deletions

File tree

‎community/modules/internal/slurm-gcp/login/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@
44
| Name | Version |
55
|------|---------|
66
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
7-
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 4.84 |
7+
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 6.41 |
88

99
## Providers
1010

1111
| Name | Version |
1212
|------|---------|
13-
| <a name="provider_google"></a> [google](#provider\_google) | >= 4.84 |
13+
| <a name="provider_google"></a> [google](#provider\_google) | >= 6.41 |
1414

1515
## Modules
1616

‎community/modules/internal/slurm-gcp/login/main.tf‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -85,9 +85,10 @@ resource "google_storage_bucket_object" "startup_scripts" {
8585
) => s.content
8686
}
8787

88-
bucket = var.slurm_bucket_name
89-
name = "${var.slurm_bucket_dir}/${each.key}"
90-
content = each.value
88+
bucket = var.slurm_bucket_name
89+
name = "${var.slurm_bucket_dir}/${each.key}"
90+
content = each.value
91+
source_md5hash = md5(each.value)
9192
}
9293

9394
locals {
@@ -101,9 +102,10 @@ locals {
101102
}
102103

103104
resource "google_storage_bucket_object" "config" {
104-
bucket = var.slurm_bucket_name
105-
name = "${var.slurm_bucket_dir}/login_group_configs/${local.name}.yaml"
106-
content = yamlencode(local.config)
105+
bucket = var.slurm_bucket_name
106+
name = "${var.slurm_bucket_dir}/login_group_configs/${local.name}.yaml"
107+
content = yamlencode(local.config)
108+
source_md5hash = md5(yamlencode(local.config))
107109

108110
# To ensure that login group "is not ready" until all startup scripts are written down
109111
depends_on = [google_storage_bucket_object.startup_scripts]

‎community/modules/internal/slurm-gcp/login/versions.tf‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ terraform {
2020
required_providers {
2121
google = {
2222
source = "hashicorp/google"
23-
version = ">= 4.84"
23+
version = ">= 6.41"
2424
}
2525
}
2626
provider_meta "google" {

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -262,14 +262,14 @@ limitations under the License.
262262
| Name | Version |
263263
|------|---------|
264264
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
265-
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 4.84 |
265+
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 6.41 |
266266
| <a name="requirement_google-beta"></a> [google-beta](#requirement\_google-beta) | >= 6.0.0 |
267267

268268
## Providers
269269

270270
| Name | Version |
271271
|------|---------|
272-
| <a name="provider_google"></a> [google](#provider\_google) | >= 4.84 |
272+
| <a name="provider_google"></a> [google](#provider\_google) | >= 6.41 |
273273
| <a name="provider_google-beta"></a> [google-beta](#provider\_google-beta) | >= 6.0.0 |
274274

275275
## Modules

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/modules/slurm_files/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ limitations under the License.
1919
|------|---------|
2020
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | ~> 1.3 |
2121
| <a name="requirement_archive"></a> [archive](#requirement\_archive) | ~> 2.0 |
22-
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 3.53 |
22+
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 6.41 |
2323
| <a name="requirement_local"></a> [local](#requirement\_local) | ~> 2.0 |
2424
| <a name="requirement_random"></a> [random](#requirement\_random) | ~> 3.0 |
2525

@@ -28,7 +28,7 @@ limitations under the License.
2828
| Name | Version |
2929
|------|---------|
3030
| <a name="provider_archive"></a> [archive](#provider\_archive) | ~> 2.0 |
31-
| <a name="provider_google"></a> [google](#provider\_google) | >= 3.53 |
31+
| <a name="provider_google"></a> [google](#provider\_google) | >= 6.41 |
3232
| <a name="provider_local"></a> [local](#provider\_local) | ~> 2.0 |
3333
| <a name="provider_random"></a> [random](#provider\_random) | ~> 3.0 |
3434

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/modules/slurm_files/main.tf‎

Lines changed: 52 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,10 @@ locals {
119119
}
120120

121121
resource "google_storage_bucket_object" "config" {
122-
bucket = data.google_storage_bucket.this.name
123-
name = "${local.bucket_dir}/config.yaml"
124-
content = yamlencode(local.config)
122+
bucket = data.google_storage_bucket.this.name
123+
name = "${local.bucket_dir}/config.yaml"
124+
content = yamlencode(local.config)
125+
source_md5hash = md5(yamlencode(local.config))
125126

126127
# Take dependency on all other "config artifacts" so creation of `config.yaml`
127128
# can be used as a signal for setup.py that "everything is ready".
@@ -141,25 +142,28 @@ resource "google_storage_bucket_object" "nodeset_config" {
141142
instance_properties = jsondecode(ns.instance_properties_json)
142143
}) }
143144

144-
bucket = data.google_storage_bucket.this.name
145-
name = "${local.bucket_dir}/nodeset_configs/${each.key}.yaml"
146-
content = yamlencode(each.value)
145+
bucket = data.google_storage_bucket.this.name
146+
name = "${local.bucket_dir}/nodeset_configs/${each.key}.yaml"
147+
content = yamlencode(each.value)
148+
source_md5hash = md5(yamlencode(each.value))
147149
}
148150

149151
resource "google_storage_bucket_object" "nodeset_dyn_config" {
150152
for_each = { for ns in var.nodeset_dyn : ns.nodeset_name => ns }
151153

152-
bucket = data.google_storage_bucket.this.name
153-
name = "${local.bucket_dir}/nodeset_dyn_configs/${each.key}.yaml"
154-
content = yamlencode(each.value)
154+
bucket = data.google_storage_bucket.this.name
155+
name = "${local.bucket_dir}/nodeset_dyn_configs/${each.key}.yaml"
156+
content = yamlencode(each.value)
157+
source_md5hash = md5(yamlencode(each.value))
155158
}
156159

157160
resource "google_storage_bucket_object" "nodeset_tpu_config" {
158161
for_each = { for n in var.nodeset_tpu[*].nodeset : n.nodeset_name => n }
159162

160-
bucket = data.google_storage_bucket.this.name
161-
name = "${local.bucket_dir}/nodeset_tpu_configs/${each.key}.yaml"
162-
content = yamlencode(each.value)
163+
bucket = data.google_storage_bucket.this.name
164+
name = "${local.bucket_dir}/nodeset_tpu_configs/${each.key}.yaml"
165+
content = yamlencode(each.value)
166+
source_md5hash = md5(yamlencode(each.value))
163167
}
164168

165169
#########
@@ -245,15 +249,17 @@ data "archive_file" "slurm_gcp_devel_compute_zip" {
245249
}
246250

247251
resource "google_storage_bucket_object" "devel" {
248-
bucket = var.bucket_name
249-
name = local.slurm_gcp_devel_zip_bucket
250-
source = data.archive_file.slurm_gcp_devel_controller_zip.output_path
252+
bucket = var.bucket_name
253+
name = local.slurm_gcp_devel_zip_bucket
254+
source = data.archive_file.slurm_gcp_devel_controller_zip.output_path
255+
source_md5hash = data.archive_file.slurm_gcp_devel_controller_zip.output_md5
251256
}
252257

253258
resource "google_storage_bucket_object" "devel_compute" {
254-
bucket = var.bucket_name
255-
name = local.slurm_gcp_devel_compute_zip_bucket
256-
source = data.archive_file.slurm_gcp_devel_compute_zip.output_path
259+
bucket = var.bucket_name
260+
name = local.slurm_gcp_devel_compute_zip_bucket
261+
source = data.archive_file.slurm_gcp_devel_compute_zip.output_path
262+
source_md5hash = data.archive_file.slurm_gcp_devel_compute_zip.output_md5
257263
}
258264

259265
###########
@@ -266,9 +272,10 @@ resource "google_storage_bucket_object" "controller_startup_scripts" {
266272
: replace(basename(x.filename), "/[^a-zA-Z0-9-_]/", "_") => x
267273
}
268274

269-
bucket = var.bucket_name
270-
name = format("%s/slurm-controller-script-%s", local.bucket_dir, each.key)
271-
content = each.value.content
275+
bucket = var.bucket_name
276+
name = format("%s/slurm-controller-script-%s", local.bucket_dir, each.key)
277+
content = each.value.content
278+
source_md5hash = md5(each.value.content)
272279
}
273280

274281
resource "google_storage_bucket_object" "nodeset_startup_scripts" {
@@ -280,9 +287,10 @@ resource "google_storage_bucket_object" "nodeset_startup_scripts" {
280287
name = format("slurm-nodeset-%s-script-%s", nodeset, replace(basename(s.filename), "/[^a-zA-Z0-9-_]/", "_")) }
281288
]]) : x.name => x.content }
282289

283-
bucket = var.bucket_name
284-
name = format("%s/%s", local.bucket_dir, each.key)
285-
content = each.value
290+
bucket = var.bucket_name
291+
name = format("%s/%s", local.bucket_dir, each.key)
292+
content = each.value
293+
source_md5hash = md5(each.value)
286294
}
287295

288296
resource "google_storage_bucket_object" "prolog_scripts" {
@@ -291,10 +299,11 @@ resource "google_storage_bucket_object" "prolog_scripts" {
291299
: replace(basename(x.filename), "/[^a-zA-Z0-9-_]/", "_") => x
292300
}
293301

294-
bucket = var.bucket_name
295-
name = format("%s/slurm-prolog-script-%s", local.bucket_dir, each.key)
296-
content = each.value.content
297-
source = each.value.source
302+
bucket = var.bucket_name
303+
name = format("%s/slurm-prolog-script-%s", local.bucket_dir, each.key)
304+
content = each.value.content
305+
source = each.value.source
306+
source_md5hash = each.value.content != null && each.value.content != "" ? md5(each.value.content) : filemd5(each.value.source)
298307
}
299308

300309
resource "google_storage_bucket_object" "epilog_scripts" {
@@ -303,10 +312,11 @@ resource "google_storage_bucket_object" "epilog_scripts" {
303312
: replace(basename(x.filename), "/[^a-zA-Z0-9-_]/", "_") => x
304313
}
305314

306-
bucket = var.bucket_name
307-
name = format("%s/slurm-epilog-script-%s", local.bucket_dir, each.key)
308-
content = each.value.content
309-
source = each.value.source
315+
bucket = var.bucket_name
316+
name = format("%s/slurm-epilog-script-%s", local.bucket_dir, each.key)
317+
content = each.value.content
318+
source = each.value.source
319+
source_md5hash = each.value.content != null && each.value.content != "" ? md5(each.value.content) : filemd5(each.value.source)
310320
}
311321

312322
resource "google_storage_bucket_object" "task_prolog_scripts" {
@@ -315,10 +325,11 @@ resource "google_storage_bucket_object" "task_prolog_scripts" {
315325
: replace(basename(x.filename), "/[^a-zA-Z0-9-_]/", "_") => x
316326
}
317327

318-
bucket = var.bucket_name
319-
name = format("%s/slurm-task_prolog-script-%s", local.bucket_dir, each.key)
320-
content = each.value.content
321-
source = each.value.source
328+
bucket = var.bucket_name
329+
name = format("%s/slurm-task_prolog-script-%s", local.bucket_dir, each.key)
330+
content = each.value.content
331+
source = each.value.source
332+
source_md5hash = each.value.content != null && each.value.content != "" ? md5(each.value.content) : filemd5(each.value.source)
322333
}
323334

324335
resource "google_storage_bucket_object" "task_epilog_scripts" {
@@ -327,10 +338,11 @@ resource "google_storage_bucket_object" "task_epilog_scripts" {
327338
: replace(basename(x.filename), "/[^a-zA-Z0-9-_]/", "_") => x
328339
}
329340

330-
bucket = var.bucket_name
331-
name = format("%s/slurm-task_epilog-script-%s", local.bucket_dir, each.key)
332-
content = each.value.content
333-
source = each.value.source
341+
bucket = var.bucket_name
342+
name = format("%s/slurm-task_epilog-script-%s", local.bucket_dir, each.key)
343+
content = each.value.content
344+
source = each.value.source
345+
source_md5hash = each.value.content != null && each.value.content != "" ? md5(each.value.content) : filemd5(each.value.source)
334346
}
335347

336348
############################

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/modules/slurm_files/versions.tf‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ terraform {
2323
}
2424
google = {
2525
source = "hashicorp/google"
26-
version = ">= 3.53"
26+
version = ">= 6.41"
2727
}
2828
random = {
2929
source = "hashicorp/random"

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/partition.tf‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -162,9 +162,10 @@ module "nodeset_cleanup_tpu" {
162162
resource "google_storage_bucket_object" "parition_config" {
163163
for_each = { for p in var.partitions : p.partition_name => p }
164164

165-
bucket = module.slurm_files.bucket_name
166-
name = "${module.slurm_files.bucket_dir}/partition_configs/${each.key}.yaml"
167-
content = yamlencode(each.value)
165+
bucket = module.slurm_files.bucket_name
166+
name = "${module.slurm_files.bucket_dir}/partition_configs/${each.key}.yaml"
167+
content = yamlencode(each.value)
168+
source_md5hash = md5(yamlencode(each.value))
168169
}
169170

170171
moved {

‎community/modules/scheduler/schedmd-slurm-gcp-v6-controller/versions.tf‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ terraform {
2020
required_providers {
2121
google = {
2222
source = "hashicorp/google"
23-
version = ">= 4.84"
23+
version = ">= 6.41"
2424
}
2525
google-beta = {
2626
source = "hashicorp/google-beta"

‎modules/scripts/startup-script/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -283,15 +283,15 @@ limitations under the License.
283283
| Name | Version |
284284
|------|---------|
285285
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.5 |
286-
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 3.83 |
286+
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 6.41 |
287287
| <a name="requirement_local"></a> [local](#requirement\_local) | >= 2.0.0 |
288288
| <a name="requirement_random"></a> [random](#requirement\_random) | ~> 3.0 |
289289
290290
## Providers
291291
292292
| Name | Version |
293293
|------|---------|
294-
| <a name="provider_google"></a> [google](#provider\_google) | >= 3.83 |
294+
| <a name="provider_google"></a> [google](#provider\_google) | >= 6.41 |
295295
| <a name="provider_local"></a> [local](#provider\_local) | >= 2.0.0 |
296296
| <a name="provider_random"></a> [random](#provider\_random) | ~> 3.0 |
297297

0 commit comments

Comments
 (0)