Skip to content

Commit 078a284

Browse files
okrauseokrause
committed
adds support for Google Cloud NetApp Volumes as new file-system
Adds modules netapp-pool and netapp-volumes as well as an example blueprint and documentation.
1 parent 532e68f commit 078a284

20 files changed

Lines changed: 1350 additions & 2 deletions

File tree

‎docs/network_storage.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ storage.
55

66
The Toolkit contains modules that will **provision**:
77

8+
- [Google Cloud NetApp Volumes (GCP managed enterprise NFS and SMB)][netapp-volumes]
89
- [Filestore (GCP managed NFS)][filestore]
910
- [DDN EXAScaler lustre][ddn-exascaler] (Deprecated, removal on July 1, 2025)
1011
- [Managed Lustre][managed-lustre]
@@ -106,6 +107,7 @@ nfs-server | via USE | via USE | via USE | via STARTUP | via USE | via USE
106107
cloud-storage-bucket (GCS)| via USE | via USE | via USE | via STARTUP | via USE | via USE
107108
DDN EXAScaler lustre | via USE | via USE | via USE | Needs Testing | via USE | via USE
108109
Managed Lustre | via USE | Needs Testing | via USE | Needs Testing | Needs Testing | Needs Testing
110+
netapp-volume | Needs Testing | Needs Testing | via USE | Needs Testing | Needs Testing | Needs Testing
109111
|  |   |   |   |   |  
110112
filestore (pre-existing) | via USE | via USE | via USE | via STARTUP | via USE | via USE
111113
nfs-server (pre-existing) | via USE | via USE | via USE | via STARTUP | via USE | via USE
@@ -129,3 +131,4 @@ GCS FUSE (pre-existing) | via USE | via USE | via USE | via STARTUP | via USE |
129131
[ddn-exascaler]: ../community/modules/file-system/DDN-EXAScaler/README.md
130132
[managed-lustre]: ../modules/file-system/managed-lustre/README.md
131133
[nfs-server]: ../community/modules/file-system/nfs-server/README.md
134+
[netapp-volumes]: ../modules/file-system/netapp-volume/README.md

‎examples/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,7 @@ md_toc github examples/README.md | sed -e "s/\s-\s/ * /"
6464
* [xpk-n2-filestore](#xpk-n2-filestore--) ![community-badge] ![experimental-badge]
6565
* [gke-h4d](#gke-h4d-) ![core-badge]
6666
* [gke-g4](#gke-g4-) ![core-badge]
67+
* [netapp-volumes.yaml](#netapp-volumesyaml--) ![core-badge]
6768
* [Blueprint Schema](#blueprint-schema)
6869
* [Writing an HPC Blueprint](#writing-an-hpc-blueprint)
6970
* [Blueprint Boilerplate](#blueprint-boilerplate)
@@ -1642,6 +1643,61 @@ This blueprint uses GKE to provision a Kubernetes cluster and a G4 node pool, al
16421643

16431644
[gke-g4]: ../examples/gke-g4
16441645

1646+
### [netapp-volumes.yaml] ![core-badge]
1647+
1648+
This blueprint demonstrates how to provision NFS volumes as shared filesystems for compute VMs, using Google Cloud NetApp Volumes. It can be used as an alternative to FileStore in blueprints.
1649+
1650+
NetApp Volumes is a first-party Google service that provides NFS and/or SMB shared file-systems to VMs. It offers advanced data management capabilities and highly scalable capacity and performance.
1651+
1652+
NetApp Volume provides:
1653+
1654+
* robust support for NFSv3, NFSv4.x and SMB 2.1 and 3.x
1655+
* a [rich feature set][service-levels]
1656+
* scalable [performance](https://cloud.google.com/netapp/volumes/docs/performance/performance-benchmarks)
1657+
* FlexCache: Caching of ONTAP-based volumes to provide high-throughput and low latency read access to compute clusters of on-premises data
1658+
* [Auto-tiering](https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/manage-auto-tiering) of unused data to optimse cost
1659+
1660+
Support for NetApp Volumes is split into two modules.
1661+
1662+
* **netapp-storage-pool** provisions a [storage pool](https://cloud.google.com/netapp/volumes/docs/configure-and-use/storage-pools/overview). Storage pools are pre-provisioned storage capacity containers which host volumes. A pool also defines fundamental properties of all the volumes within, like the region, the attached network, the [service level][service-levels], CMEK encryption, Active Directory and LDAP settings.
1663+
* **netapp-volume** provisions a [volume](https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/overview) inside an existing storage pool. A volume is a file-system which is shared using NFS or SMB. It provides advanced data management capabilities.
1664+
1665+
You can provision multiple volumes in a pool. For service levels Standard, Premium and Extreme the throughput capability depends on volume size and service level. Every GiB of provisioned volume space adds 16/64/128 KiBps of throughput capability.
1666+
1667+
#### Steps to deploy the blueprint
1668+
1669+
To provision the bluebrint, please run:
1670+
1671+
```shell
1672+
./gcluster create examples/netapp-volumes.yaml --vars "project_id=${GOOGLE_CLOUD_PROJECT}" --vars region=us-central1 --vars zone=us-central1-a
1673+
./gcluster deploy netapp-volumes
1674+
```
1675+
1676+
After the blueprint deployed, you can login to the VM created:
1677+
1678+
```shell
1679+
gcloud compute ssh --zone "us-central1-a" "netapp-volumes-0" --project ${GOOGLE_CLOUD_PROJECT} --tunnel-through-iap
1680+
```
1681+
1682+
A NetApp Volumes volume is provisioned and mounted to /home in all the provisioned VMs. A home directory for your user is created automatically:
1683+
1684+
```shell
1685+
pwd
1686+
df -h -t nfs
1687+
```
1688+
1689+
#### Clean Up
1690+
To destroy all resources associated with creating the GKE cluster, run the following command:
1691+
1692+
```sh
1693+
./gcluster destroy netapp-volumes
1694+
```
1695+
1696+
[netapp-storage-pool]: ../netapp-storage-pool/README.md
1697+
[service-levels]: https://cloud.google.com/netapp/volumes/docs/discover/service-levels
1698+
[auto-tiering]: https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/manage-auto-tiering
1699+
[netapp-volumes.yaml]: ../examples/netapp-volumes.yaml
1700+
16451701
## Blueprint Schema
16461702

16471703
Similar documentation can be found on

‎examples/netapp-volumes.yaml‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
---
16+
17+
# This blueprint show how to provision shared file systems with Google Cloud NetApp Volumes.
18+
# It creates a NetApp storage pool and a volume for use by VM instances.
19+
# It can be used to build compute clusters on top of it and as a drop-in replacement
20+
# for Filestore in existing blueprints.
21+
22+
blueprint_name: netapp-volumes
23+
24+
vars:
25+
project_id: ## Set GCP Project ID Here ##
26+
deployment_name: netapp-volumes
27+
region: ## Set GCP Region Here ##
28+
zone: ## Set GCP Zone Here ##
29+
pool_service_level: "EXTREME" # Options: "STANDARD", "PREMIUM", "EXTREME"
30+
31+
# Documentation for each of the modules used below can be found at
32+
# https://github.com/GoogleCloudPlatform/cluster-toolkit
33+
34+
deployment_groups:
35+
- group: primary
36+
modules:
37+
- id: network
38+
source: modules/network/vpc
39+
40+
# Private Service Access (PSA) requires the compute.networkAdmin role which is
41+
# included in the Owner role, but not Editor.
42+
# https://cloud.google.com/vpc/docs/configure-private-services-access#permissions
43+
- id: private_service_access
44+
source: community/modules/network/private-service-access
45+
use: [network]
46+
settings:
47+
prefix_length: 24
48+
service_name: "netapp.servicenetworking.goog"
49+
deletion_policy: "ABANDON"
50+
51+
- id: netapp_pool
52+
source: modules/file-system/netapp-storage-pool
53+
use: [network, private_service_access]
54+
settings:
55+
pool_name: $(vars.deployment_name)-netapp-pool
56+
capacity_gib: 2048
57+
service_level: $(vars.pool_service_level)
58+
region: $(vars.region)
59+
# allow_auto_tiering: true
60+
61+
- id: homefs
62+
source: modules/file-system/netapp-volume
63+
use: [netapp_pool] # Create this pool using the netapp-storage-pool module
64+
settings:
65+
region: $(vars.region)
66+
volume_name: $(vars.deployment_name)-homefs
67+
capacity_gib: 2048 # Size up to available capacity in the pool
68+
large_capacity: false
69+
local_mount: "/home" # Mount point at client when client uses USE directive
70+
# mount_options: "..." # Use custom mount_options for special use cases. Defaults are sane.
71+
protocols: ["NFSV3"] # List of protocols. ["NFSV3], ["NFSv4] or ["NFSV3, "NFSV4"]
72+
unix_permissions: "0777" # Specify default permissions for root inode owned by root:root
73+
# If no export policy is specified, a permissive default policy will be applied, which is:
74+
# allowed_clients = "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16" # RFC1918
75+
# has_root_access = true # no_root_squash enabled
76+
# access_type = "READ_WRITE"
77+
# export_policy_rules:
78+
# - allowed_clients: "10.10.20.8,10.10.20.9"
79+
# has_root_access: true # no_root_squash enabled
80+
# access_type: "READ_WRITE"
81+
# nfsv3: true
82+
# nfsv4: false
83+
# - allowed_clients: "10.0.0.0/8"
84+
# has_root_access: false # no_root_squash disabled
85+
# access_type: "READ_WRITE"
86+
# nfsv3: true
87+
# nfsv4: false
88+
# tiering_policy: # Enable auto-tiering. Requires auto-tiering enabled storage pool
89+
# tier_action: "ENABLED"
90+
# cooling_threshold_days: 31 # tier data blocks which have not been touched for 31 days
91+
# description: "Shared volume for EDA job"
92+
# labels:
93+
# department: eda
94+
95+
# Example VMs which use homefs
96+
- id: gcnv_ubuntu_instances
97+
source: modules/compute/vm-instance
98+
use: [network, homefs]
99+
settings:
100+
instance_count: 1
101+
machine_type: n2-standard-2
102+
103+
- id: wait-for-vms
104+
source: community/modules/scripts/wait-for-startup
105+
settings:
106+
instance_names: $(gcnv_ubuntu_instances.name)
107+
timeout: 7200

‎modules/file-system/netapp-storage-pool/README.md‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
## Description
2+
3+
This module creates a [Google Cloud NetApp Volumes](https://cloud.google.com/netapp/volumes/docs/discover/overview)
4+
storage pool.
5+
6+
NetApp Volumes is a first-party Google service that provides NFS and/or SMB shared file-systems to VMs. It offers advanced data management capabilities and highly scalable capacity and performance.
7+
NetApp Volume provides:
8+
9+
- robust support for NFSv3, NFSv4.x and SMB 2.1 and 3.x
10+
- a [rich feature set][service-levels]
11+
- scalable [performance](https://cloud.google.com/netapp/volumes/docs/performance/performance-benchmarks)
12+
- FlexCache: Caching of ONTAP-based volumes to provide high-throughput and low latency read access to compute clusters of on-premises data
13+
- [Auto-tiering](https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/manage-auto-tiering) of unused data to optimse cost
14+
15+
Support for NetApp Volumes is split into two modules.
16+
17+
- **netapp-storage-pool** provisions a [storage pool](https://cloud.google.com/netapp/volumes/docs/configure-and-use/storage-pools/overview). Storage pools are pre-provisioned storage capacity containers which host volumes. A pool also defines fundamental properties of all the volumes within, like the region, the attached network, the [service level][service-levels], CMEK encryption, Active Directory and LDAP settings.
18+
- **netapp-volume** provisions a [volume](https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/overview) inside an existing storage pool. A volume file-system container which is shared using NFS or SMB. It provides advanced data management capabilities.
19+
20+
For more information on this and other network storage options in the Cluster
21+
Toolkit, see the extended [Network Storage documentation](../../../docs/network_storage.md).
22+
23+
### NetApp storage pool service levels
24+
25+
The netapp-storage-pool module currently supports the following NetApp Volumes [service levels][service-levels]:
26+
27+
- Standard: 16 KiBps throughput per provisioned KiB of volume capacity.
28+
- Premium: 64 KiBps throughput per provisioned KiB of volume capacity. Optional [auto-tiering].
29+
- Extreme: 128 KiBps throughput per provisioned KiB of volume capacity. Optional [auto-tiering].
30+
31+
Check the [service level matrix][service-levels] for additional information on capability differences between service levels. Flex service levels are currently not supported, but you can connect to existing Flex volumes using the [pre-existing-network-storage module][pre-existing].
32+
33+
### On-boarding NetApp Volumes
34+
NetApp Volumes uses [Private Service Access](https://cloud.google.com/vpc/docs/private-services-access) (PSA) to connect volumes to your network. Before you create a storage pool, make sure to [connect NetApp Volumes to your network](https://cloud.google.com/netapp/volumes/docs/get-started/configure-access/networking).
35+
36+
Example of creating a storage pool using a new network:
37+
38+
```yaml
39+
deployment_groups:
40+
- group: primary
41+
modules:
42+
- id: network
43+
source: modules/network/vpc
44+
settings:
45+
region: $(vars.region)
46+
47+
- id: private_service_access
48+
source: community/modules/network/private-service-access
49+
use: [network]
50+
settings:
51+
prefix_length: 24
52+
service_name: "netapp.servicenetworking.goog"
53+
deletion_policy: "ABANDON"
54+
55+
- id: netapp_pool
56+
source: modules/file-system/netapp-storage-pool
57+
use: [network, private_service_access]
58+
settings:
59+
pool_name: $(vars.deployment_name)-eda-pool
60+
capacity_gib: 20000
61+
service_level: "EXTREME"
62+
region: $(vars.region)
63+
```
64+
65+
Example of creating a storage pool using an existing network which was already PSA-peered with NetApp Volume:
66+
67+
```yaml
68+
deployment_groups:
69+
- group: primary
70+
modules:
71+
- id: network
72+
source: modules/network/pre-existing-vpc
73+
settings:
74+
project_id: $(vars.project_id)
75+
region: $(vars.region)
76+
network_name: $(vars.network)
77+
78+
- id: netapp_pool
79+
source: modules/file-system/netapp-storage-pool
80+
use: [network]
81+
settings:
82+
pool_name: "eda-pool"
83+
capacity_gib: 20000
84+
service_level: "EXTREME"
85+
region: $(vars.region)
86+
```
87+
88+
### Storage pool example
89+
90+
The following example shows all available parameters in use:
91+
92+
```yaml
93+
- id: netapp_pool
94+
source: modules/file-system/netapp-storage-pool
95+
use: [network, private_service_access]
96+
settings:
97+
pool_name: "mypool"
98+
region: "us-west4"
99+
capacity_gib: 2048
100+
service_level: "EXTREME"
101+
active_directory_policy: "projects/myproject/locations/us-east4/activeDirectories/my-ad"
102+
cmek_policy: "projects/myproject/locations/us-east4/kmsConfigs/my-cmek-policy"
103+
ldap_enabled: false
104+
allow_auto_tiering: false
105+
description: "Demo storage pool"
106+
labels:
107+
owner: bob
108+
```
109+
110+
### NetApp Volumes quota
111+
112+
Your project must have unused quota for NetApp Volumes in the region you will
113+
provision the storage pool. This can be found by browsing to the [Quota tab within IAM & Admin](https://console.cloud.google.com/iam-admin/quotas) in the Cloud Console.
114+
Please note that there are separate quota limits for Standard and Premium/Extreme service levels.
115+
116+
See also NetApp Volumes [default quotas](https://cloud.google.com/netapp/volumes/docs/quotas#netapp-volumes-default-quotas).
117+
118+
[service-levels]: https://cloud.google.com/netapp/volumes/docs/discover/service-levels
119+
[auto-tiering]: https://cloud.google.com/netapp/volumes/docs/configure-and-use/volumes/manage-auto-tiering
120+
[pre-existing]: ../pre-existing-network-storage/README.md
121+
[matrix]: ../../../docs/network_storage.md#compatibility-matrix
122+
123+
## License
124+
125+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
126+
Copyright 2025 Google LLC
127+
128+
Licensed under the Apache License, Version 2.0 (the "License");
129+
you may not use this file except in compliance with the License.
130+
You may obtain a copy of the License at
131+
132+
http://www.apache.org/licenses/LICENSE-2.0
133+
134+
Unless required by applicable law or agreed to in writing, software
135+
distributed under the License is distributed on an "AS IS" BASIS,
136+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
137+
See the License for the specific language governing permissions and
138+
limitations under the License.
139+
140+
## Requirements
141+
142+
| Name | Version |
143+
|------|---------|
144+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.5.7 |
145+
| <a name="requirement_google"></a> [google](#requirement\_google) | >= 6.45.0 |
146+
| <a name="requirement_random"></a> [random](#requirement\_random) | ~> 3.0 |
147+
148+
## Providers
149+
150+
| Name | Version |
151+
|------|---------|
152+
| <a name="provider_google"></a> [google](#provider\_google) | >= 6.45.0 |
153+
| <a name="provider_random"></a> [random](#provider\_random) | ~> 3.0 |
154+
155+
## Modules
156+
157+
No modules.
158+
159+
## Resources
160+
161+
| Name | Type |
162+
|------|------|
163+
| [google_netapp_storage_pool.netapp_storage_pool](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/netapp_storage_pool) | resource |
164+
| [random_id.resource_name_suffix](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource |
165+
| [google_compute_network_peering.private_peering](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_network_peering) | data source |
166+
167+
## Inputs
168+
169+
| Name | Description | Type | Default | Required |
170+
|------|-------------|------|---------|:--------:|
171+
| <a name="input_active_directory_policy"></a> [active\_directory\_policy](#input\_active\_directory\_policy) | The ID of the Active Directory policy to apply to the storage pool in the format:<br/>`projects/<project_id>/locations/<location>/activeDirectoryPolicies/<policy_id>` | `string` | `null` | no |
172+
| <a name="input_allow_auto_tiering"></a> [allow\_auto\_tiering](#input\_allow\_auto\_tiering) | Whether to allow automatic tiering for the storage pool. | `bool` | `false` | no |
173+
| <a name="input_capacity_gib"></a> [capacity\_gib](#input\_capacity\_gib) | The capacity of the storage pool in GiB. | `number` | `2048` | no |
174+
| <a name="input_cmek_policy"></a> [cmek\_policy](#input\_cmek\_policy) | The ID of the Customer Managed Encryption Key (CMEK) policy to apply to the storage pool in the format:<br/>`projects/<project>/locations/<location>/kmsConfigs/<name>` | `string` | `null` | no |
175+
| <a name="input_deployment_name"></a> [deployment\_name](#input\_deployment\_name) | Name of the deployment, used as name of the NetApp storage pool if no name is specified. | `string` | n/a | yes |
176+
| <a name="input_description"></a> [description](#input\_description) | A description of the NetApp storage pool. | `string` | `""` | no |
177+
| <a name="input_labels"></a> [labels](#input\_labels) | Labels to add to the NetApp storage pool. Key-value pairs. | `map(string)` | n/a | yes |
178+
| <a name="input_ldap_enabled"></a> [ldap\_enabled](#input\_ldap\_enabled) | Whether to enable LDAP for the storage pool. | `bool` | `false` | no |
179+
| <a name="input_network_id"></a> [network\_id](#input\_network\_id) | The ID of the GCE VPC network to which the NetApp storage pool is connected given in the format:<br/>`projects/<project_id>/global/networks/<network_name>`" | `string` | n/a | yes |
180+
| <a name="input_network_self_link"></a> [network\_self\_link](#input\_network\_self\_link) | Network self-link the pool will be on, required for checking private service access | `string` | n/a | yes |
181+
| <a name="input_pool_name"></a> [pool\_name](#input\_pool\_name) | The name of the storage pool. Leave empty to generate name based on deployment name. | `string` | `null` | no |
182+
| <a name="input_private_vpc_connection_peering"></a> [private\_vpc\_connection\_peering](#input\_private\_vpc\_connection\_peering) | The name of the private VPC connection peering. | `string` | `"sn-netapp-prod"` | no |
183+
| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | ID of project in which the NetApp storage pool will be created. | `string` | n/a | yes |
184+
| <a name="input_region"></a> [region](#input\_region) | Location for NetApp storage pool. | `string` | n/a | yes |
185+
| <a name="input_service_level"></a> [service\_level](#input\_service\_level) | The service level of the storage pool. | `string` | `"PREMIUM"` | no |
186+
187+
## Outputs
188+
189+
| Name | Description |
190+
|------|-------------|
191+
| <a name="output_capacity_gb"></a> [capacity\_gb](#output\_capacity\_gb) | Storage pool capacity in GiB. |
192+
| <a name="output_netapp_storage_pool_id"></a> [netapp\_storage\_pool\_id](#output\_netapp\_storage\_pool\_id) | An identifier for the resource with format `projects/{{project}}/locations/{{location}}/storagePools/{{name}}` |
193+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

0 commit comments

Comments
 (0)