Adding IPV6 & IDPF support (GoogleCloudPlatform#5066)

Author: LAVEEN

modules/compute/vm-instance/README.md

@@ -226,7 +226,7 @@ limitations under the License.
 | <a name="input_metadata"></a> [metadata](#input\_metadata) | Metadata, provided as a map | `map(string)` | `{}` | no |
 | <a name="input_min_cpu_platform"></a> [min\_cpu\_platform](#input\_min\_cpu\_platform) | The name of the minimum CPU platform that you want the instance to use. | `string` | `null` | no |
 | <a name="input_name_prefix"></a> [name\_prefix](#input\_name\_prefix) | An optional name for all VM and disk resources.<br/>If not supplied, `deployment_name` will be used.<br/>When `name_prefix` is supplied, and `add_deployment_name_before_prefix` is set,<br/>then resources are named by "<`deployment_name`>-<`name_prefix`>-<#>". | `string` | `null` | no |
-| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | A list of network interfaces. The options match that of the terraform<br/>network\_interface block of google\_compute\_instance. For descriptions of the<br/>subfields or more information see the documentation:<br/>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance#nested_network_interface<br/><br/>**\_NOTE:\_** If `network_interfaces` are set, `network_self_link` and<br/>`subnetwork_self_link` will be ignored, even if they are provided through<br/>the `use` field. `bandwidth_tier` and `disable_public_ips` also do not apply<br/>to network interfaces defined in this variable.<br/><br/>Subfields:<br/>network            (string, required if subnetwork is not supplied)<br/>subnetwork         (string, required if network is not supplied)<br/>subnetwork\_project (string, optional)<br/>network\_ip         (string, optional)<br/>nic\_type           (string, optional, choose from ["GVNIC", "VIRTIO\_NET", "MRDMA", "IRDMA"])<br/>stack\_type         (string, optional, choose from ["IPV4\_ONLY", "IPV4\_IPV6"])<br/>queue\_count        (number, optional)<br/>access\_config      (object, optional)<br/>ipv6\_access\_config (object, optional)<br/>alias\_ip\_range     (list(object), optional) | <pre>list(object({<br/>    network            = string,<br/>    subnetwork         = string,<br/>    subnetwork_project = string,<br/>    network_ip         = string,<br/>    nic_type           = string,<br/>    stack_type         = string,<br/>    queue_count        = number,<br/>    access_config = list(object({<br/>      nat_ip                 = string,<br/>      public_ptr_domain_name = string,<br/>      network_tier           = string<br/>    })),<br/>    ipv6_access_config = list(object({<br/>      public_ptr_domain_name = string,<br/>      network_tier           = string<br/>    })),<br/>    alias_ip_range = list(object({<br/>      ip_cidr_range         = string,<br/>      subnetwork_range_name = string<br/>    }))<br/>  }))</pre> | `[]` | no |
+| <a name="input_network_interfaces"></a> [network\_interfaces](#input\_network\_interfaces) | A list of network interfaces. The options match that of the terraform<br/>network\_interface block of google\_compute\_instance. For descriptions of the<br/>subfields or more information see the documentation:<br/>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance#nested_network_interface<br/><br/>**\_NOTE:\_** If `network_interfaces` are set, `network_self_link` and<br/>`subnetwork_self_link` will be ignored, even if they are provided through<br/>the `use` field. `bandwidth_tier` and `disable_public_ips` also do not apply<br/>to network interfaces defined in this variable.<br/><br/>Subfields:<br/>network            (string, required if subnetwork is not supplied)<br/>subnetwork         (string, required if network is not supplied)<br/>subnetwork\_project (string, optional)<br/>network\_ip         (string, optional)<br/>nic\_type           (string, optional, choose from ["GVNIC", "VIRTIO\_NET", "MRDMA", "IRDMA", "IDPF"])<br/>stack\_type         (string, optional, choose from ["IPV4\_ONLY", "IPV4\_IPV6", "IPV6\_ONLY"])<br/>queue\_count        (number, optional)<br/>access\_config      (object, optional)<br/>ipv6\_access\_config (object, optional)<br/>alias\_ip\_range     (list(object), optional) | <pre>list(object({<br/>    network            = string,<br/>    subnetwork         = string,<br/>    subnetwork_project = string,<br/>    network_ip         = string,<br/>    nic_type           = string,<br/>    stack_type         = string,<br/>    queue_count        = number,<br/>    access_config = list(object({<br/>      nat_ip                 = string,<br/>      public_ptr_domain_name = string,<br/>      network_tier           = string<br/>    })),<br/>    ipv6_access_config = list(object({<br/>      public_ptr_domain_name = string,<br/>      network_tier           = string<br/>    })),<br/>    alias_ip_range = list(object({<br/>      ip_cidr_range         = string,<br/>      subnetwork_range_name = string<br/>    }))<br/>  }))</pre> | `[]` | no |
 | <a name="input_network_self_link"></a> [network\_self\_link](#input\_network\_self\_link) | The self link of the network to attach the VM. Can use "default" for the default network. | `string` | `null` | no |
 | <a name="input_network_storage"></a> [network\_storage](#input\_network\_storage) | An array of network attached storage mounts to be configured. | <pre>list(object({<br/>    server_ip             = string,<br/>    remote_mount          = string,<br/>    local_mount           = string,<br/>    fs_type               = string,<br/>    mount_options         = string,<br/>    client_install_runner = map(string)<br/>    mount_runner          = map(string)<br/>  }))</pre> | `[]` | no |
 | <a name="input_on_host_maintenance"></a> [on\_host\_maintenance](#input\_on\_host\_maintenance) | Describes maintenance behavior for the instance. If left blank this will default to `MIGRATE` except for when `placement_policy`, spot provisioning, or GPUs require it to be `TERMINATE` | `string` | `null` | no |

modules/compute/vm-instance/variables.tf

@@ -195,8 +195,8 @@ variable "network_interfaces" {
     subnetwork         (string, required if network is not supplied)
     subnetwork_project (string, optional)
     network_ip         (string, optional)
-    nic_type           (string, optional, choose from ["GVNIC", "VIRTIO_NET", "MRDMA", "IRDMA"])
-    stack_type         (string, optional, choose from ["IPV4_ONLY", "IPV4_IPV6"])
+    nic_type           (string, optional, choose from ["GVNIC", "VIRTIO_NET", "MRDMA", "IRDMA", "IDPF"])
+    stack_type         (string, optional, choose from ["IPV4_ONLY", "IPV4_IPV6", "IPV6_ONLY"])
     queue_count        (number, optional)
     access_config      (object, optional)
     ipv6_access_config (object, optional)
@@ -233,15 +233,15 @@ variable "network_interfaces" {
   }
   validation {
     condition = alltrue([
-      for ni in var.network_interfaces : ni.nic_type == "GVNIC" || ni.nic_type == "VIRTIO_NET" || ni.nic_type == "MRDMA" || ni.nic_type == "IRDMA" || ni.nic_type == null
+      for ni in var.network_interfaces : ni.nic_type == null || contains(["GVNIC", "VIRTIO_NET", "MRDMA", "IRDMA", "IDPF"], ni.nic_type)
     ])
-    error_message = "In the variable network_interfaces, field \"nic_type\" must be \"GVNIC\", \"VIRTIO_NET\", \"MRDMA\", \"IRDMA\", or null."
+    error_message = "In the variable network_interfaces, field \"nic_type\" must be \"GVNIC\", \"VIRTIO_NET\", \"MRDMA\", \"IRDMA\", \"IDPF\" or null."
   }
   validation {
     condition = alltrue([
-      for ni in var.network_interfaces : ni.stack_type == "IPV4_ONLY" || ni.stack_type == "IPV4_IPV6" || ni.stack_type == null
+      for ni in var.network_interfaces : ni.stack_type == null || contains(["IPV4_ONLY", "IPV4_IPV6", "IPV6_ONLY"], ni.stack_type)
     ])
-    error_message = "In the variable network_interfaces, field \"stack_type\" must be either \"IPV4_ONLY\", \"IPV4_IPV6\" or null."
+    error_message = "In the variable network_interfaces, field \"stack_type\" must be either \"IPV4_ONLY\", \"IPV4_IPV6\", \"IPV6_ONLY\" or null."
   }
 }
 

modules/network/gpu-rdma-vpc/README.md

@@ -97,7 +97,9 @@ limitations under the License.
 
 ## Providers
 
-No providers.
+| Name | Version |
+|------|---------|
+| <a name="provider_terraform"></a> [terraform](#provider\_terraform) | n/a |
 
 ## Modules
 
@@ -107,7 +109,9 @@ No providers.
 
 ## Resources
 
-No resources.
+| Name | Type |
+|------|------|
+| [terraform_data.network_profile_validation](https://registry.terraform.io/providers/hashicorp/terraform/latest/docs/resources/data) | resource |
 
 ## Inputs
 

modules/network/gpu-rdma-vpc/main.tf

@@ -17,10 +17,15 @@
 locals {
   autoname      = replace(var.deployment_name, "_", "-")
   network_name  = var.network_name == null ? "${local.autoname}-net" : var.network_name
-  subnet_prefix = var.subnetworks_template.name_prefix == null ? "${local.autoname}-subnet" : var.subnetworks_template.name_prefix
+  subnet_prefix = var.subnetworks_template == null || var.subnetworks_template.name_prefix == null ? "${local.autoname}-subnet" : var.subnetworks_template.name_prefix
+  is_roce_metal = var.network_profile != null ? can(regex("vpc-roce-metal", var.network_profile)) : false
 
-  new_bits = ceil(log(var.subnetworks_template.count, 2))
-  template_subnetworks = [for i in range(var.subnetworks_template.count) :
+  new_bits = local.is_roce_metal ? 0 : ceil(log(var.subnetworks_template.count, 2))
+  /** RoCE metal profile doesn't allow creating a subnetwork, as the profile 
+  * automatically creates a predefined subnetwork for the MRDMA interfaces.
+  * See: https://cloud.google.com/vpc/docs/network-profiles
+  **/
+  template_subnetworks = local.is_roce_metal ? [] : [for i in range(var.subnetworks_template.count) :
     {
       subnet_name   = "${local.subnet_prefix}-${i}"
       subnet_region = try(var.subnetworks_template.region, var.region)
@@ -77,3 +82,16 @@ module "vpc" {
   firewall_rules                         = local.firewall_rules
   network_profile                        = var.network_profile
 }
+
+resource "terraform_data" "network_profile_validation" {
+  lifecycle {
+    precondition {
+      condition = (
+        can(regex("vpc-roce-metal", var.network_profile)) ?
+        var.subnetworks_template == null :
+        var.subnetworks_template != null
+      )
+      error_message = "subnetworks_template must be null when using 'vpc-roce-metal' network profile and non-null for all other profiles."
+    }
+  }
+}

modules/network/gpu-rdma-vpc/outputs.tf

@@ -55,5 +55,6 @@ output "subnetwork_interfaces_gke" {
 
 output "subnetwork_name_prefix" {
   description = "Prefix of the RDMA subnetwork names"
-  value       = var.subnetworks_template.name_prefix
+  # This uses a conditional: if the variable is null, return null; otherwise, return the prefix.
+  value = var.subnetworks_template != null ? var.subnetworks_template.name_prefix : null
 }

modules/network/gpu-rdma-vpc/variables.tf

@@ -50,7 +50,7 @@ variable "subnetworks_template" {
   ip_range    (string, required, range of IPs for all subnets to share (CIDR format), default is 192.168.0.0/16)
   region      (string, optional, region to deploy subnets to, defaults to vars.region)
   EOT
-  nullable    = false
+  nullable    = true
   type = object({
     count       = number
     name_prefix = string
@@ -65,12 +65,15 @@ variable "subnetworks_template" {
   }
 
   validation {
-    condition     = var.subnetworks_template.count > 0
-    error_message = "Number of subnetworks must be greater than 0"
+    # If template is provided, count must be > 0
+    condition     = var.subnetworks_template == null ? true : var.subnetworks_template.count > 0
+    error_message = "Number of subnetworks must be greater than 0."
   }
 
   validation {
-    condition     = can(cidrhost(var.subnetworks_template.ip_range, 0))
+    # If the template is null, return true (pass). 
+    # Otherwise, check if the CIDR format is valid.
+    condition     = var.subnetworks_template == null ? true : can(cidrhost(var.subnetworks_template.ip_range, 0))
     error_message = "IP address range must be in CIDR format."
   }
 }

modules/network/vpc/README.md

@@ -200,9 +200,11 @@ limitations under the License.
 | <a name="input_enable_iap_ssh_ingress"></a> [enable\_iap\_ssh\_ingress](#input\_enable\_iap\_ssh\_ingress) | Enable a firewall rule to allow SSH access using IAP tunnels | `bool` | `true` | no |
 | <a name="input_enable_iap_winrm_ingress"></a> [enable\_iap\_winrm\_ingress](#input\_enable\_iap\_winrm\_ingress) | Enable a firewall rule to allow Windows Remote Management (WinRM) access using IAP tunnels | `bool` | `false` | no |
 | <a name="input_enable_internal_traffic"></a> [enable\_internal\_traffic](#input\_enable\_internal\_traffic) | Enable a firewall rule to allow all internal TCP, UDP, and ICMP traffic within the network | `bool` | `true` | no |
+| <a name="input_enable_ipv6_ula"></a> [enable\_ipv6\_ula](#input\_enable\_ipv6\_ula) | Enable IPv6 ULA, this is a permanent change and cannot be undone! | `bool` | `false` | no |
 | <a name="input_extra_iap_ports"></a> [extra\_iap\_ports](#input\_extra\_iap\_ports) | A list of TCP ports for which to create firewall rules that enable IAP for TCP forwarding (use dedicated enable\_iap variables for standard ports) | `list(string)` | `[]` | no |
 | <a name="input_firewall_log_config"></a> [firewall\_log\_config](#input\_firewall\_log\_config) | Firewall log configuration for Toolkit firewall rules (var.enable\_iap\_ssh\_ingress and others) | `string` | `"DISABLE_LOGGING"` | no |
 | <a name="input_firewall_rules"></a> [firewall\_rules](#input\_firewall\_rules) | List of firewall rules | `any` | `[]` | no |
+| <a name="input_internal_ipv6_range"></a> [internal\_ipv6\_range](#input\_internal\_ipv6\_range) | When enabling IPv6 ULA, optionally specify a /48 from fd20::/20 (default null) | `string` | `null` | no |
 | <a name="input_ips_per_nat"></a> [ips\_per\_nat](#input\_ips\_per\_nat) | The number of IP addresses to allocate for each regional Cloud NAT (set to 0 to disable NAT). The number of NAT IPs depend on the port reservation allocated for each node and the number of ports that a single NAT IP can serve. Refer this documentation for more details: https://cloud.google.com/nat/docs/ports-and-addresses#port-reservation-examples | `number` | `2` | no |
 | <a name="input_labels"></a> [labels](#input\_labels) | Labels to add to network resources that support labels. Key-value pairs of strings. | `map(string)` | `{}` | no |
 | <a name="input_mtu"></a> [mtu](#input\_mtu) | The network MTU (default: 8896). Recommended values: 0 (use Compute Engine default), 1460 (default outside HPC environments), 1500 (Internet default), or 8896 (for Jumbo packets). Allowed are all values in the range 1300 to 8896, inclusively. | `number` | `8896` | no |

modules/network/vpc/main.tf

@@ -191,6 +191,8 @@ module "vpc" {
   secondary_ranges                       = length(local.secondary_ranges_map) > 0 ? local.secondary_ranges_map : var.secondary_ranges
   routing_mode                           = var.network_routing_mode
   mtu                                    = var.mtu
+  enable_ipv6_ula                        = var.enable_ipv6_ula
+  internal_ipv6_range                    = var.internal_ipv6_range
   description                            = var.network_description
   shared_vpc_host                        = var.shared_vpc_host
   delete_default_internet_gateway_routes = var.delete_default_internet_gateway_routes

modules/network/vpc/variables.tf

@@ -82,6 +82,18 @@ variable "mtu" {
   default     = 8896
 }
 
+variable "enable_ipv6_ula" {
+  description = "Enable IPv6 ULA, this is a permanent change and cannot be undone!"
+  type        = bool
+  default     = false
+}
+
+variable "internal_ipv6_range" {
+  description = "When enabling IPv6 ULA, optionally specify a /48 from fd20::/20 (default null)"
+  type        = string
+  default     = null
+}
+
 variable "subnetworks" {
   description = <<-EOT
   List of subnetworks to create within the VPC. If left empty, it will be