Skip to content

Commit 709500e

Browse files
shmueldananshmueldanan
authored
Merge pull request #266 from NetApp/add_export_fsxn_cf
Add export fsxn to cf program.
2 parents b6e711d + 0751113 commit 709500e

File tree

3 files changed

+267
-8
lines changed

3 files changed

+267
-8
lines changed

‎CloudFormation/Export-FSxN-CloudFormation/README.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Export NetApp FSxN to a CloudFormation Template
2+
3+
## Overview
4+
This folder provides a script that will create a CloudFormation template based on the current configuration of an existing FSx for ONTAP file system.
5+
6+
## Prerequisites
7+
- An FSxN file system you want to create a CloudFormation template for.
8+
- An AWS account with permissions to "describe" the FSxN file system and its virtual storage machines, and volumes.
9+
- The AWS CLI installed and configured on your local machine. You can find instructions on how to do that [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html).
10+
11+
## Running the script
12+
13+
The script takes the following parameters:
14+
- `-f fs-id`: The ID of the FSxN file system you want to create the CloudFormation template for. This is a required parameter.
15+
- `-n name`: Is an optional name to be appended to all the volumes, svms and NetBIOS names. This is so you can test the CloudFormation template while the original machine is still running.
16+
17+
The script will output the CloudFormation template in JSON format. You can redirect this output to a file if you want to save it.
18+
19+
Note that since you can't retrieve credentials from the FSxN configuration the script will create
20+
parameters that will allow you to provide an AWS Secrets Manager secret that should contain the credentials.
21+
There will be one parameter for the password of the 'fsxadmin' account. That secret will just need one 'key'
22+
named "password" with the desired fsxadmin password. There will also be a parameter for each SVMs that has an
23+
Active Directory configured for it so you can provide a secret that should have a 'username' and 'password' key
24+
that will be used to join the SVM to the domain.
25+
26+
An example run:
27+
```
28+
$ python export_fsxn_cf.py -f fs-0123456789abcdef0 -n test > fsxn_template.json
29+
Warning: Volume rvnw_vol_autogrow does not have a junction path yet it is required for a Cloudformation template so setting it to /rvnw_vol_autogrow
30+
Warning: Volume unixdata does not have a junction path yet it is required for a Cloudformation template so setting it to /unixdata
31+
Warning: Volume effictest2 is a DP volume and cannot have the StorageEfficiencyEnabled property, removing it from the CloudFormation template.
32+
Warning: Volume effictest2 is a DP volume and cannot have the SnapshotPolicy property, removing it from the CloudFormation template.
33+
Warning: Volume effictest2 is a DP volume and cannot have the SecurityStyle property, removing it from the CloudFormation template.
34+
Warning: Could not find root volume for SVM fsa. Setting the security style to UNIX
35+
```
36+
37+
## Notes
38+
- For multi availability zone deployments, the script will do the following in regards to the Endpoint IP Address Range:
39+
- If the file system is in the 198.19.0.0/16 address range (the AWS default), the script will not provide an address range forcing AWS to just allocate a new address range from the 198.19.0.0/16 CIDR block.
40+
- If it isn't in the 198.19.0.0/16 address range then it will create a parameter so you can specify a new address range for testing purposes, with a default set to the current address range.
41+
- Since AWS requires you to provide a junction path when creating a volume, if the script finds a volume without a junction path it will set it to `/volume_name`. A warning message will be outputted if this happens to alert you.
42+
- Since AWS doesn't allow you to specify these parameters when creating a DP type volume, their current settings will be removed from the CloudFormation template:
43+
- SecurityStyle
44+
- SnapshotPolicy
45+
- StorageEfficiencyEnabled
46+
- If, for some reason, the script can't find the attributes of the root volume of a SVM (unlikely but there are reasons how this can happen), it will set the security style of the SVM to 'NTFS' if the SVM has a Active Directory configuration, otherwise it will set it to a 'UNIX' security style. A warning message will be outputted if this happens to alert you.
47+
- Since AWS only allows an Active Directory Distinguished Name (DN) to start with "OU=", if the script finds a DN that doesn't start with "OU=" it will ignore it and will output a warning message to alert you. However the DN is set to "CN=Computers", which is the default DN that ONTAP will use when joining a domain to AD, it will not output an warning message with the assumption that by not providing a DN it will be set to "CN=Computers".
48+
- While some testing was performed, hence the `-n` option, not for all possible FSxN configurations were tested. If you run into any issues with the script, or have suggestions for improvements, please open an [issue](https://github.com/NetApp/FSx-ONTAP-samples-scripts/issues) on GitHub.
49+
50+
## Author Information
51+
52+
This repository is maintained by the contributors listed on [GitHub](https://github.com/NetApp/FSx-ONTAP-samples-scripts/graphs/contributors).
53+
54+
## License
55+
56+
Licensed under the Apache License, Version 2.0 (the "License").
57+
58+
You may obtain a copy of the License at [apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).
59+
60+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an _"AS IS"_ basis, without WARRANTIES or conditions of any kind, either express or implied.
61+
62+
See the License for the specific language governing permissions and limitations under the License.
63+
64+
© 2025 NetApp, Inc. All Rights Reserved.

‎CloudFormation/Export-FSxN-CloudFormation/export_fsxn_cf.py‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
#!/bin/python3
2+
#
3+
# This script takes an FSx for ONTAP file system ID as input and generates
4+
# a CloudFormation template for the file system, its volumes, and its
5+
# storage virtual machines. The output is printed to the console in JSON format.
6+
################################################################################
7+
8+
import json
9+
import boto3
10+
import sys
11+
import optparse
12+
#
13+
# Get the file system ID from the command line
14+
parser = optparse.OptionParser()
15+
parser.add_option('-f', dest='filesystemId', help='The ID of the FSx for ONTAP file system to generate the CloudFormation template for.')
16+
parser.add_option('-n', dest='nameAppend', help='A string to append to the names of the resources in the CloudFormation template to make them unique. This is optional.')
17+
opts, args = parser.parse_args()
18+
19+
if opts.filesystemId is None:
20+
print("Error: -f option is required", file=sys.stderr)
21+
sys.exit(1)
22+
filesystemId = opts.filesystemId
23+
nameAppend = opts.nameAppend if opts.nameAppend is not None else ""
24+
#
25+
# Create boto3 client for fsx and ec2.
26+
fsxClient = boto3.client('fsx')
27+
ec2Client = boto3.client('ec2')
28+
#
29+
# Get the file system details
30+
response = fsxClient.describe_file_systems(FileSystemIds=[filesystemId])
31+
if response.get('FileSystems') is None or len(response['FileSystems']) == 0:
32+
print(f"No file system found with ID {filesystemId}", file=sys.stderr)
33+
sys.exit(1)
34+
#
35+
# Build the CloudFormation template for the file system
36+
cfTemplate = {
37+
"Description": f"FSx File System template for {filesystemId}.",
38+
"Resources": {}
39+
}
40+
cfTemplate['Parameters'] = {
41+
"fsxadminPassword": {
42+
"Type": "String",
43+
"Description": "The AWS Secrets Manager secret that has the password for the fsxadmin user. It should have a key named 'password' that contains the password."
44+
}
45+
}
46+
fileSystem = response['FileSystems'][0]
47+
fsCfTemplate = {}
48+
fsCfTemplate['Type'] = 'AWS::FSx::FileSystem'
49+
fsCfTemplate['Properties'] = {}
50+
for prop in ['NetworkType', 'FileSystemType', 'KmsKeyId', 'StorageCapacity', 'SubnetIds', 'StorageType', 'Tags']:
51+
if prop in fileSystem:
52+
fsCfTemplate['Properties'][prop] = fileSystem[prop]
53+
#
54+
# Get the security groups from the ENIs.
55+
fsCfTemplate['Properties']['SecurityGroupIds'] = []
56+
securityGroups = {} # Use a dictionary to store the security groups to avoid duplicates.
57+
response = ec2Client.describe_network_interfaces(NetworkInterfaceIds=fileSystem['NetworkInterfaceIds'])
58+
for eni in response['NetworkInterfaces']:
59+
for group in eni['Groups']:
60+
securityGroups[group['GroupId']] = 1
61+
for group in securityGroups.keys():
62+
fsCfTemplate['Properties']['SecurityGroupIds'].append(group)
63+
#
64+
# Copy the ONTAP configuration.
65+
fsCfTemplate['Properties']['OntapConfiguration'] = {"FsxAdminPassword": {"Fn::Sub": "{{resolve:secretsmanager:${fsxadminPassword}:SecretString:password}}"}}
66+
for prop in ['AutomaticBackupRetentionDays', 'DailyAutomaticBackupStartTime', 'DeploymentType',
67+
'EndpointIpAddressRange', 'EndpointIpv6AddressRange', 'PreferredSubnetId', 'RouteTableIds',
68+
'WeeklyMaintenanceStartTime', 'HAPairs', 'ThroughputCapacityPerHAPair']:
69+
if prop in fileSystem['OntapConfiguration']:
70+
fsCfTemplate['Properties']['OntapConfiguration'][prop] = fileSystem['OntapConfiguration'][prop]
71+
if fileSystem['OntapConfiguration']['DiskIopsConfiguration']['Mode'] == 'AUTOMATIC':
72+
fsCfTemplate['Properties']['OntapConfiguration']['DiskIopsConfiguration'] = {'Mode': 'AUTOMATIC'}
73+
else:
74+
fsCfTemplate['Properties']['OntapConfiguration']['DiskIopsConfiguration'] = fileSystem['OntapConfiguration']['DiskIopsConfiguration']
75+
#
76+
# If using the default endpoint IP address range, remove it from the
77+
# CloudFormation template since AWS will automatically use a new default
78+
# address range if the 'EndpointIpAddressRange' is not specified.
79+
if 'EndpointIpAddressRange' in fsCfTemplate['Properties']['OntapConfiguration']:
80+
if fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange'].startswith("198.19"):
81+
del fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange']
82+
else:
83+
cfTemplate['Parameters']['endpointIpAddressRange'] = {
84+
"Type": "String",
85+
"Description": "The IP address range to use for the file system's endpoints.",
86+
"Default": fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange']
87+
}
88+
fsCfTemplate['Properties']['OntapConfiguration']['EndpointIpAddressRange'] = {"Ref": "endpointIpAddressRange"}
89+
90+
cfTemplate['Resources'].update({filesystemId.replace("-", ""): fsCfTemplate})
91+
#
92+
# Get all the volumes for the file system. Getting the volumes before the SVMs
93+
# since I need the list of volumes to get the security style of the root volume
94+
# for each SVMs.
95+
response = fsxClient.describe_volumes(Filters=[{'Name': 'file-system-id', 'Values': [filesystemId]}])
96+
volumes = response['Volumes']
97+
for volume in volumes:
98+
if volume['OntapConfiguration']['StorageVirtualMachineRoot']:
99+
continue
100+
volumeCfTemplate = {}
101+
volumeCfTemplate['Type'] = 'AWS::FSx::Volume'
102+
volumeCfTemplate['Properties'] = {}
103+
for property in ['Name', 'VolumeType', 'Tags']:
104+
if property in volume:
105+
volumeCfTemplate['Properties'][property] = volume[property]
106+
volumeCfTemplate['Properties']['Name'] = volumeCfTemplate['Properties']['Name'] + nameAppend
107+
108+
volumeCfTemplate['Properties']['OntapConfiguration'] = {}
109+
for property in ['AggregateName', 'CopyTagsToBackups', 'OntapVolumeType', 'SecurityStyle', 'SizeInMegabytes',
110+
'StorageEfficiencyEnabled', 'VolumeStyle', 'JunctionPath',
111+
'SnapshotPolicy', 'TieringPolicy', 'SnaplockConfiguration']:
112+
if property in volume['OntapConfiguration']:
113+
volumeCfTemplate['Properties']['OntapConfiguration'][property] = volume['OntapConfiguration'][property]
114+
volumeCfTemplate['Properties']['OntapConfiguration']['StorageVirtualMachineId'] = {"Ref" : volume['OntapConfiguration']['StorageVirtualMachineId'].replace("-", "")}
115+
#
116+
# DP volumes can't have JunctionPath, StorageEfficiency, SnapshotPolicy or SecurityStyle properties
117+
if volume['OntapConfiguration']['OntapVolumeType'] == 'DP':
118+
for prop in ['JunctionPath', 'StorageEfficiencyEnabled', 'SnapshotPolicy', 'SecurityStyle']:
119+
if prop in volumeCfTemplate['Properties']['OntapConfiguration']:
120+
print(f"Warning: Volume {volume['Name']} is a DP volume and cannot have the {prop} property, removing it from the CloudFormation template.", file=sys.stderr)
121+
del volumeCfTemplate['Properties']['OntapConfiguration'][prop]
122+
else:
123+
if 'JunctionPath' not in volumeCfTemplate['Properties']['OntapConfiguration']:
124+
print(f"Warning: Volume {volume['Name']} does not have a junction path yet it is required for a CloudFormation template so setting it to /{volumeCfTemplate['Properties']['Name']}", file=sys.stderr)
125+
volumeCfTemplate['Properties']['OntapConfiguration']['JunctionPath'] = "/" + volumeCfTemplate['Properties']['Name']
126+
127+
cfTemplate['Resources'].update({volume['VolumeId'].replace("-", ""): volumeCfTemplate})
128+
#
129+
# Get all the storage virtual machines for the file system.
130+
response = fsxClient.describe_storage_virtual_machines(Filters=[{'Name': 'file-system-id', 'Values': [filesystemId]}])
131+
for svm in response['StorageVirtualMachines']:
132+
svmCfTemplate = {}
133+
svmCfTemplate['Type'] = 'AWS::FSx::StorageVirtualMachine'
134+
svmCfTemplate['Properties'] = {"FileSystemId": {"Ref" : filesystemId.replace("-", "")}}
135+
for prop in ['ActiveDirectoryConfiguration', 'Name', 'RootVolumeSecurityStyle', 'Tags']:
136+
if prop in svm:
137+
svmCfTemplate['Properties'][prop] = svm[prop]
138+
svmCfTemplate['Properties']['Name'] = svmCfTemplate['Properties']['Name'] + nameAppend
139+
140+
if 'ActiveDirectoryConfiguration' in svm:
141+
if len(svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName']) > 10 and len(nameAppend) > 0:
142+
svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'][:10] + nameAppend.upper()
143+
else:
144+
svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] + nameAppend.upper()
145+
svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'] = svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['NetBiosName'][:15]
146+
if 'SelfManagedActiveDirectoryConfiguration' in svm['ActiveDirectoryConfiguration']:
147+
if 'OrganizationalUnitDistinguishedName' in svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']:
148+
#
149+
# Since CF can only handle organizational unit distinguish names that have a
150+
# parent of OU, we need to check if the parent of the organizational unit is
151+
# OU and if not, we need to remove the organizational unit distinguish name (DN)
152+
# from the CloudFormation template and print a warning message.
153+
dnParent=svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName'].split(",")[0]
154+
dnParent = dnParent.split("=")[0]
155+
if dnParent != "OU":
156+
#
157+
# The default value from ONTAP is 'CN=Computers' which does not have a
158+
# parent of OU, but CF requires that the parent is OU, therefore we will
159+
# just ignore the organizational unit distinguish name.
160+
if svm['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName'] != "CN=Computers":
161+
print(f'Warning: The organizational unit distinguish name for the SVM {svm["Name"]} is "{svm["ActiveDirectoryConfiguration"]["SelfManagedActiveDirectoryConfiguration"]["OrganizationalUnitDistinguishedName"]}" which does not have a parent of a OU and CloudFormation requires that, therefore the distinguished name is ignored. This will cause the SVM to be put into the "default" computer location', file=sys.stderr)
162+
del svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['OrganizationalUnitDistinguishedName']
163+
164+
secretParameterId = f'{svm["Name"].replace("-", "").replace("_", "")}AdminCredentials'
165+
cfTemplate['Parameters'][secretParameterId] = {
166+
"Type": "String",
167+
"Description": f"The AWS Secrets Manager secret that has the Active Directory credentials for the {svm['Name']} storage virtual machine. It should have two keys named 'username' and 'password'."
168+
}
169+
svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['UserName'] = {"Fn::Sub": "{{resolve:secretsmanager:${" + secretParameterId + "}:SecretString:username}}"}
170+
svmCfTemplate['Properties']['ActiveDirectoryConfiguration']['SelfManagedActiveDirectoryConfiguration']['Password'] = {"Fn::Sub": "{{resolve:secretsmanager:${" + secretParameterId + "}:SecretString:password}}"}
171+
#
172+
# Get the security style for the SVM's root volume. Assume the root volume is <svm_name>_root
173+
for volume in volumes:
174+
if volume['OntapConfiguration']['StorageVirtualMachineId'] == svm['StorageVirtualMachineId'] and volume['OntapConfiguration']['StorageVirtualMachineRoot']:
175+
svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = volume['OntapConfiguration']['SecurityStyle']
176+
break
177+
if svmCfTemplate['Properties'].get('RootVolumeSecurityStyle') is None:
178+
if 'ActiveDirectoryConfiguration' in svmCfTemplate['Properties']:
179+
svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = 'NTFS'
180+
else:
181+
svmCfTemplate['Properties']['RootVolumeSecurityStyle'] = 'UNIX'
182+
print(f"Warning: Could not find root volume for SVM {svm['Name']}. Setting the security style to {svmCfTemplate['Properties']['RootVolumeSecurityStyle']}.", file=sys.stderr)
183+
cfTemplate['Resources'].update({svm['StorageVirtualMachineId'].replace("-", ""): svmCfTemplate})
184+
185+
cfTemplate['Outputs'] = {
186+
"FileSystemId": {
187+
"Description": "The ID of the FSx for ONTAP file system.",
188+
"Value": {"Ref": filesystemId.replace("-", "")}
189+
}
190+
}
191+
# Print the CloudFormation template in JSON format
192+
print(json.dumps(cfTemplate, indent=4))

0 commit comments

Comments
 (0)