Skip to content

Commit 70e1797

Browse files
matthew-richersonmatthew-richerson
committed
Variable override documentation
Add documentation for the variable override section of the NnfStorageProfile. Include examples for overriding $LV_NAME. Signed-off-by: Matt Richerson <matthew.richerson@hpe.com>
1 parent 76de320 commit 70e1797

1 file changed

Lines changed: 155 additions & 0 deletions

File tree

‎docs/guides/storage-profiles/readme.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -836,6 +836,161 @@ The following variables are available to `preMountMGTCommands`:
836836
| `$JOBID` | Job ID from the Workflow |
837837
| `$FS_NAME` | Lustre file system name |
838838

839+
### Variable Override
840+
841+
The `variableOverride` fields in a storage profile allow overriding the default values of environment variables used in command lines and internally by the NNF software. This is an advanced feature that provides fine-grained control over storage configuration.
842+
843+
When a variable is overridden, the NNF software uses the new value internally when manipulating block devices and file systems; not just for expanding `$VARIABLE` references in command lines. For example, overriding `$LV_NAME` changes the actual logical volume name that the NNF software uses to perform checks and run internal commands.
844+
845+
Variables are expanded recursively so that variable values can reference other variables. This allows building values based on job, workflow, and directive identifiers in a similar way to what the NNF software does internally.
846+
847+
848+
#### Overridable Variables by File System Type
849+
850+
##### XFS, GFS2, and Raw Allocations
851+
852+
For XFS, GFS2, and Raw file systems, add `variableOverride` at the storage type level:
853+
854+
```yaml
855+
xfsStorage:
856+
variableOverride:
857+
"$VG_NAME": "custom-vg"
858+
"$LV_NAME": "custom-lv"
859+
```
860+
861+
| Variable | Default | Description |
862+
|----------|---------|-------------|
863+
| `$JOBID` | From workflow | Job ID from the workflow |
864+
| `$USERID` | From workflow | User ID of the job submitter |
865+
| `$GROUPID` | From workflow | Group ID of the job submitter |
866+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
867+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
868+
| `$VG_NAME` | Auto-generated from workflow UID | Volume group name used by LVM |
869+
| `$LV_NAME` | Auto-generated | Logical volume name used by LVM |
870+
| `$PERCENT_VG` | Calculated based on allocation | Size as percentage of volume group |
871+
| `$FSTYPE` | `xfs`, `gfs2`, or `none` | File system type passed to mkfs |
872+
| `$MOUNT_TARGET` | `directory` or `file` | Whether mount target is a file or directory |
873+
| `$TEMP_DIR` | Auto-generated | Temporary directory for operations |
874+
| `$MOUNT_PATH` | `/mnt/nnf/<workflow-uid>-<index>` | Path where file system is mounted (affects DW_JOB_* variables) |
875+
876+
##### Lustre Allocations
877+
878+
For Lustre file systems, `variableOverride` can be specified at the target level (mgtOptions, mdtOptions, mgtMdtOptions, ostOptions) or for client operations (clientOptions):
879+
880+
```yaml
881+
lustreStorage:
882+
ostOptions:
883+
variableOverride:
884+
"$ZPOOL_NAME": "custom-pool"
885+
clientOptions:
886+
variableOverride:
887+
"$MOUNT_PATH": "/lustre/scratch"
888+
```
889+
890+
**Target-level variables (mgtOptions, mdtOptions, mgtMdtOptions, ostOptions):**
891+
892+
| Variable | Default | Description |
893+
|----------|---------|-------------|
894+
| `$JOBID` | From workflow | Job ID from the workflow |
895+
| `$USERID` | From workflow | User ID of the job submitter |
896+
| `$GROUPID` | From workflow | Group ID of the job submitter |
897+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
898+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
899+
| `$ZPOOL_NAME` | Auto-generated from workflow UID | ZFS pool name |
900+
| `$ZPOOL_DATA_SET` | Target type (mgt, mdt, ost) | ZFS dataset name within pool |
901+
| `$FSNAME` | Auto-generated | Lustre file system name |
902+
| `$TARGET_TYPE` | mgt, mdt, mgtmdt, or ost | Type of Lustre target |
903+
| `$TARGET_PATH` | Auto-generated | Mount path for the target |
904+
| `$MGS_NID` | Auto-discovered | NID of the MGS |
905+
| `$BACKFS` | zfs | Backing file system type |
906+
907+
**Client-level variables (clientOptions):**
908+
909+
| Variable | Default | Description |
910+
|----------|---------|-------------|
911+
| `$JOBID` | From workflow | Job ID from the workflow |
912+
| `$USERID` | From workflow | User ID of the job submitter |
913+
| `$GROUPID` | From workflow | Group ID of the job submitter |
914+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
915+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
916+
| `$MOUNT_PATH` | `/mnt/nnf/<workflow-uid>-<index>` | Path where Lustre client is mounted (affects DW_JOB_* variables) |
917+
918+
#### Example: Custom Logical Volume Name for XFS
919+
920+
This example shows how to override the logical volume name for an XFS file system. The `$LV_NAME` variable is used both in command lines and internally by the NNF software when creating the LVM logical volume.
921+
922+
```yaml
923+
apiVersion: nnf.cray.hpe.com/v1alpha10
924+
kind: NnfStorageProfile
925+
metadata:
926+
name: custom-lv-name
927+
namespace: nnf-system
928+
data:
929+
default: false
930+
xfsStorage:
931+
variableOverride:
932+
"$LV_NAME": "shared-lv"
933+
blockDeviceCommands:
934+
rabbitCommands:
935+
pvCreate: $DEVICE
936+
pvRemove: $DEVICE
937+
vgCreate: --addtag $JOBID $VG_NAME $DEVICE_LIST
938+
vgRemove: $VG_NAME
939+
lvCreate: --zero n --activate n --size $LV_SIZE --stripes $DEVICE_NUM --stripesize=32KiB --name $LV_NAME $VG_NAME
940+
lvRemove: $VG_NAME/$LV_NAME
941+
lvChange:
942+
activate: --activate y $VG_NAME/$LV_NAME
943+
deactivate: --activate n $VG_NAME/$LV_NAME
944+
computeCommands:
945+
lvChange:
946+
activate: --activate y $VG_NAME/$LV_NAME
947+
deactivate: --activate n $VG_NAME/$LV_NAME
948+
fileSystemCommands:
949+
rabbitCommands:
950+
mkfs: $DEVICE
951+
mount: $DEVICE $MOUNT_PATH
952+
unmount: $MOUNT_PATH
953+
computeCommands:
954+
mount: $DEVICE $MOUNT_PATH
955+
unmount: $MOUNT_PATH
956+
```
957+
958+
#### Example: Custom Mount Path with DW_JOB_* Integration
959+
960+
The `$MOUNT_PATH` variable is special because it affects the path that gets reported to the workload manager in the `DW_JOB_*` environment variables. This example shows a custom mount path that will be visible to user jobs.
961+
962+
```yaml
963+
apiVersion: nnf.cray.hpe.com/v1alpha10
964+
kind: NnfStorageProfile
965+
metadata:
966+
name: custom-mount-path
967+
namespace: nnf-system
968+
data:
969+
default: false
970+
xfsStorage:
971+
variableOverride:
972+
"$MOUNT_PATH": "/scratch/$JOBID"
973+
# ... rest of configuration
974+
```
975+
976+
#### Example: Recursive Variable Expansion
977+
978+
Variables can reference other variables, which are expanded recursively. This is useful for building paths that include system-provided values:
979+
980+
```yaml
981+
apiVersion: nnf.cray.hpe.com/v1alpha10
982+
kind: NnfStorageProfile
983+
metadata:
984+
name: recursive-vars
985+
namespace: nnf-system
986+
data:
987+
default: false
988+
xfsStorage:
989+
variableOverride:
990+
"$MOUNT_PATH": "/scratch/$WORKFLOW_UID/$DIRECTIVE_INDEX"
991+
# ... rest of configuration
992+
```
993+
839994
## Advanced Configuration
840995

841996
### External MGS

0 commit comments

Comments
 (0)