Skip to content

Commit 9cbed6b

Browse files
matthew-richersonmatthew-richerson
authored
Variable override documentation (#301)
* Variable override documentation Add documentation for the variable override section of the NnfStorageProfile. Include examples for overriding $LV_NAME. Signed-off-by: Matt Richerson <matthew.richerson@hpe.com> * describe new variables Signed-off-by: Matt Richerson <matthew.richerson@hpe.com> --------- Signed-off-by: Matt Richerson <matthew.richerson@hpe.com>
1 parent 76de320 commit 9cbed6b

1 file changed

Lines changed: 157 additions & 0 deletions

File tree

‎docs/guides/storage-profiles/readme.md‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -636,6 +636,8 @@ Available in all commands:
636636
| `$JOBID` | Job ID from the Workflow |
637637
| `$USERID` | User ID of the job submitter |
638638
| `$GROUPID` | Group ID of the job submitter |
639+
| `$ALLOCATION_COUNT` | Number of allocations on this Rabbit for this directive |
640+
| `$HOST_COUNT` | Total host count: compute nodes assigned to a Rabbit plus the Rabbit itself |
639641

640642
### LVM Variables
641643

@@ -836,6 +838,161 @@ The following variables are available to `preMountMGTCommands`:
836838
| `$JOBID` | Job ID from the Workflow |
837839
| `$FS_NAME` | Lustre file system name |
838840

841+
### Variable Override
842+
843+
The `variableOverride` fields in a storage profile allow overriding the default values of environment variables used in command lines and internally by the NNF software. This is an advanced feature that provides fine-grained control over storage configuration.
844+
845+
When a variable is overridden, the NNF software uses the new value internally when manipulating block devices and file systems; not just for expanding `$VARIABLE` references in command lines. For example, overriding `$LV_NAME` changes the actual logical volume name that the NNF software uses to perform checks and run internal commands.
846+
847+
Variables are expanded recursively so that variable values can reference other variables. This allows building values based on job, workflow, and directive identifiers in a similar way to what the NNF software does internally.
848+
849+
850+
#### Overridable Variables by File System Type
851+
852+
##### XFS, GFS2, and Raw Allocations
853+
854+
For XFS, GFS2, and Raw file systems, add `variableOverride` at the storage type level:
855+
856+
```yaml
857+
xfsStorage:
858+
variableOverride:
859+
"$VG_NAME": "custom-vg"
860+
"$LV_NAME": "custom-lv"
861+
```
862+
863+
| Variable | Default | Description |
864+
|----------|---------|-------------|
865+
| `$JOBID` | From workflow | Job ID from the workflow |
866+
| `$USERID` | From workflow | User ID of the job submitter |
867+
| `$GROUPID` | From workflow | Group ID of the job submitter |
868+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
869+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
870+
| `$VG_NAME` | Auto-generated from workflow UID | Volume group name used by LVM |
871+
| `$LV_NAME` | Auto-generated | Logical volume name used by LVM |
872+
| `$PERCENT_VG` | Calculated based on allocation | Size as percentage of volume group |
873+
| `$FSTYPE` | `xfs`, `gfs2`, or `none` | File system type passed to mkfs |
874+
| `$MOUNT_TARGET` | `directory` or `file` | Whether mount target is a file or directory |
875+
| `$TEMP_DIR` | Auto-generated | Temporary directory for operations |
876+
| `$MOUNT_PATH` | `/mnt/nnf/<workflow-uid>-<index>` | Path where file system is mounted (affects DW_JOB_* variables) |
877+
878+
##### Lustre Allocations
879+
880+
For Lustre file systems, `variableOverride` can be specified at the target level (mgtOptions, mdtOptions, mgtMdtOptions, ostOptions) or for client operations (clientOptions):
881+
882+
```yaml
883+
lustreStorage:
884+
ostOptions:
885+
variableOverride:
886+
"$ZPOOL_NAME": "custom-pool"
887+
clientOptions:
888+
variableOverride:
889+
"$MOUNT_PATH": "/lustre/scratch"
890+
```
891+
892+
**Target-level variables (mgtOptions, mdtOptions, mgtMdtOptions, ostOptions):**
893+
894+
| Variable | Default | Description |
895+
|----------|---------|-------------|
896+
| `$JOBID` | From workflow | Job ID from the workflow |
897+
| `$USERID` | From workflow | User ID of the job submitter |
898+
| `$GROUPID` | From workflow | Group ID of the job submitter |
899+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
900+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
901+
| `$ZPOOL_NAME` | Auto-generated from workflow UID | ZFS pool name |
902+
| `$ZPOOL_DATA_SET` | Target type (mgt, mdt, ost) | ZFS dataset name within pool |
903+
| `$FSNAME` | Auto-generated | Lustre file system name |
904+
| `$TARGET_TYPE` | mgt, mdt, mgtmdt, or ost | Type of Lustre target |
905+
| `$TARGET_PATH` | Auto-generated | Mount path for the target |
906+
| `$MGS_NID` | Auto-discovered | NID of the MGS |
907+
| `$BACKFS` | zfs | Backing file system type |
908+
909+
**Client-level variables (clientOptions):**
910+
911+
| Variable | Default | Description |
912+
|----------|---------|-------------|
913+
| `$JOBID` | From workflow | Job ID from the workflow |
914+
| `$USERID` | From workflow | User ID of the job submitter |
915+
| `$GROUPID` | From workflow | Group ID of the job submitter |
916+
| `$WORKFLOW_UID` | From workflow | Kubernetes UID of the workflow resource |
917+
| `$DIRECTIVE_INDEX` | From workflow | Index of the directive within the workflow |
918+
| `$MOUNT_PATH` | `/mnt/nnf/<workflow-uid>-<index>` | Path where Lustre client is mounted (affects DW_JOB_* variables) |
919+
920+
#### Example: Custom Logical Volume Name for XFS
921+
922+
This example shows how to override the logical volume name for an XFS file system. The `$LV_NAME` variable is used both in command lines and internally by the NNF software when creating the LVM logical volume.
923+
924+
```yaml
925+
apiVersion: nnf.cray.hpe.com/v1alpha10
926+
kind: NnfStorageProfile
927+
metadata:
928+
name: custom-lv-name
929+
namespace: nnf-system
930+
data:
931+
default: false
932+
xfsStorage:
933+
variableOverride:
934+
"$LV_NAME": "shared-lv"
935+
blockDeviceCommands:
936+
rabbitCommands:
937+
pvCreate: $DEVICE
938+
pvRemove: $DEVICE
939+
vgCreate: --addtag $JOBID $VG_NAME $DEVICE_LIST
940+
vgRemove: $VG_NAME
941+
lvCreate: --zero n --activate n --size $LV_SIZE --stripes $DEVICE_NUM --stripesize=32KiB --name $LV_NAME $VG_NAME
942+
lvRemove: $VG_NAME/$LV_NAME
943+
lvChange:
944+
activate: --activate y $VG_NAME/$LV_NAME
945+
deactivate: --activate n $VG_NAME/$LV_NAME
946+
computeCommands:
947+
lvChange:
948+
activate: --activate y $VG_NAME/$LV_NAME
949+
deactivate: --activate n $VG_NAME/$LV_NAME
950+
fileSystemCommands:
951+
rabbitCommands:
952+
mkfs: $DEVICE
953+
mount: $DEVICE $MOUNT_PATH
954+
unmount: $MOUNT_PATH
955+
computeCommands:
956+
mount: $DEVICE $MOUNT_PATH
957+
unmount: $MOUNT_PATH
958+
```
959+
960+
#### Example: Custom Mount Path with DW_JOB_* Integration
961+
962+
The `$MOUNT_PATH` variable is special because it affects the path that gets reported to the workload manager in the `DW_JOB_*` environment variables. This example shows a custom mount path that will be visible to user jobs.
963+
964+
```yaml
965+
apiVersion: nnf.cray.hpe.com/v1alpha10
966+
kind: NnfStorageProfile
967+
metadata:
968+
name: custom-mount-path
969+
namespace: nnf-system
970+
data:
971+
default: false
972+
xfsStorage:
973+
variableOverride:
974+
"$MOUNT_PATH": "/scratch/$JOBID"
975+
# ... rest of configuration
976+
```
977+
978+
#### Example: Recursive Variable Expansion
979+
980+
Variables can reference other variables, which are expanded recursively. This is useful for building paths that include system-provided values:
981+
982+
```yaml
983+
apiVersion: nnf.cray.hpe.com/v1alpha10
984+
kind: NnfStorageProfile
985+
metadata:
986+
name: recursive-vars
987+
namespace: nnf-system
988+
data:
989+
default: false
990+
xfsStorage:
991+
variableOverride:
992+
"$MOUNT_PATH": "/scratch/$WORKFLOW_UID/$DIRECTIVE_INDEX"
993+
# ... rest of configuration
994+
```
995+
839996
## Advanced Configuration
840997

841998
### External MGS

0 commit comments

Comments
 (0)