Release v0.1.26

docs/guides/ha-cluster/readme.md

@@ -15,7 +15,7 @@ NNF software supports provisioning of Red Hat GFS2 (Global File System 2) storag
 - [Quorum Configuration](#quorum-configuration)
   - [Why the Rabbit Needs 17 Votes](#why-the-rabbit-needs-17-votes)
   - [Configuring Quorum Votes](#configuring-quorum-votes)
-- [Fencing with fence_recorder](#fencing-with-fence_recorder)
+- [Fencing with fence_dummy](#fencing-with-fence_dummy)
   - [How It Works](#how-it-works)
   - [Installation](#installation)
   - [STONITH Configuration](#stonith-configuration)
@@ -169,9 +169,9 @@ corosync-quorumtool
 
 The output should show the Rabbit with 17 votes and each compute with 1 vote, for a total of 33 votes and quorum at 17.
 
-## Fencing with fence_recorder
+## Fencing with fence_dummy
 
-The `fence_recorder` agent coordinates fencing with external NNF software using a request/response file pattern. When Pacemaker decides to fence a compute node, `fence_recorder`:
+The [`fence_dummy`](https://github.com/ClusterLabs/fence-agents/tree/main/agents/dummy) agent coordinates fencing with external NNF software using a request/response file pattern. When Pacemaker decides to fence a compute node, `fence_dummy`:
 
 1. Writes a fence request file
 2. Waits for the NNF software to process the request and write a response
@@ -188,10 +188,10 @@ This allows the NNF software to perform storage cleanup (detaching NVMe namespac
 │  Pacemaker/Corosync  │
 │  (Cluster Manager)   │
 └──────────┬───────────┘
-           │ Calls fence_recorder
+           │ Calls fence_dummy
            ▼
 ┌──────────────────────┐       ┌───────────────────────────────┐
-│   fence_recorder     │──────▶│   Request File                │
+│   fence_dummy        │──────▶│   Request File                │
 │   (Fence Agent)      │       │   requests/<node>-<uuid>.json │
 └──────────────────────┘       └───────────────────────────────┘
            │                                  │
@@ -218,12 +218,12 @@ This allows the NNF software to perform storage cleanup (detaching NVMe namespac
 
 ### Installation
 
-Install `fence_recorder` on all nodes in the cluster:
+Install `fence_dummy` on all nodes in the cluster:
 
 ```bash
 # Copy the agent to each node
-sudo cp fence_recorder /usr/sbin/fence_recorder
-sudo chmod 755 /usr/sbin/fence_recorder
+sudo cp fence_dummy /usr/sbin/fence_dummy
+sudo chmod 755 /usr/sbin/fence_dummy
 
 # Create the request/response directories on each rabbit node
 sudo mkdir -p /localdisk/fence-recorder/{requests,responses}
@@ -241,7 +241,7 @@ Create a STONITH resource for each compute node. Run these commands from any nod
 ```bash
 # Create STONITH resources for all 16 compute nodes (1-16)
 for i in $(seq 1 16); do
-    pcs stonith create compute-${i}-fence-recorder fence_recorder \
+    pcs stonith create compute-${i}-fence-dummy fence_dummy \
         port=compute-${i} \
         pcmk_host_list=compute-${i} \
         request_dir=/localdisk/fence-recorder/requests \
@@ -258,7 +258,7 @@ Alternatively, create resources individually:
 
 ```bash
 # Example: Create STONITH for rabbit-compute-1
-pcs stonith create compute-1-fence-recorder fence_recorder \
+pcs stonith create compute-1-fence-dummy fence_dummy \
     port=rabbit-compute-1 \
     pcmk_host_list=compute-1 \
     request_dir=/localdisk/fence-recorder/requests \
@@ -273,10 +273,34 @@ pcs stonith create compute-1-fence-recorder fence_recorder \
 |--------|---------|-------------|
 | `port` | (required) | Target node name to fence |
 | `pcmk_host_list` | (required) | Node this STONITH resource can fence |
-| `request_dir` | `/var/run/fence_recorder/requests` | Directory for fence request files |
-| `response_dir` | `/var/run/fence_recorder/responses` | Directory for fence response files |
+| `request_dir` | `/var/run/fence_dummy/requests` | Directory for fence request files |
+| `response_dir` | `/var/run/fence_dummy/responses` | Directory for fence response files |
 | `log_dir` | `/var/log/cluster` | Directory for log files |
 
+### NNF Software Configuration
+
+The nnf-node-manager DaemonSet must be configured with the **same** request and response directories used by the `fence_dummy` agent. These are controlled by environment variables in the pod spec:
+
+| Environment Variable | Default | Description |
+|---------------------|---------|-------------|
+| `NNF_FENCE_REQUEST_DIR` | `/localdisk/fence-recorder/requests` | Directory where fence agents write JSON request files |
+| `NNF_FENCE_RESPONSE_DIR` | `/localdisk/fence-recorder/responses` | Directory where nnf-sos writes JSON response files |
+
+These variables are set in the nnf-node-manager DaemonSet manifest (`config/manager/manager.yaml`):
+
+```yaml
+env:
+  - name: NNF_FENCE_REQUEST_DIR
+    value: "/localdisk/fence-recorder/requests"
+  - name: NNF_FENCE_RESPONSE_DIR
+    value: "/localdisk/fence-recorder/responses"
+```
+
+When unset or empty, the defaults shown above are used. The nnf-node-manager creates the directories automatically if they do not exist.
+
+!!! important
+    The `request_dir` and `response_dir` values in the STONITH resource configuration **must match** the `NNF_FENCE_REQUEST_DIR` and `NNF_FENCE_RESPONSE_DIR` environment variables in the nnf-node-manager pod. If they differ, fence requests will not be detected.
+
 ### Verifying Configuration
 
 ```bash
@@ -286,15 +310,15 @@ pcs stonith status
 # View STONITH configuration
 pcs stonith config
 
-# Test that fence_recorder can generate metadata
-fence_recorder --action metadata
+# Test that fence_dummy can generate metadata
+fence_dummy --action metadata
 ```
 
 ## Request/Response Protocol
 
 ### Request File Format
 
-When a fence operation is triggered, `fence_recorder` writes a JSON request file:
+When a fence operation is triggered, `fence_dummy` writes a JSON request file:
 
 **Location**: `<request_dir>/<node>-<uuid>.json`
 
@@ -379,21 +403,21 @@ journalctl -u pacemaker | grep -i stonith
 ls -la /localdisk/fence-recorder/requests/
 ls -la /localdisk/fence-recorder/responses/
 
-# Check fence_recorder logs
+# Check fence_dummy logs
 tail -f /var/log/cluster/fence-events.log
 ```
 
 **Module not found error:**
-Ensure the fencing library path is correct in `/usr/sbin/fence_recorder`. The `sys.path.append` line should point to `/usr/share/fence`.
+Ensure the fencing library path is correct in `/usr/sbin/fence_dummy`. The `sys.path.append` line should point to `/usr/share/fence`.
 
 ### Testing Fence Operations
 
 ```bash
 # Test metadata generation
-fence_recorder --action metadata
+fence_dummy --action metadata
 
 # Test monitor action (non-destructive)
-fence_recorder --action monitor -n rabbit-compute-1 \
+fence_dummy --action monitor -n rabbit-compute-1 \
     --request-dir=/localdisk/fence-recorder/requests \
     --response-dir=/localdisk/fence-recorder/responses
 ```

docs/repo-guides/release-nnf-sw/release-all.md

@@ -17,33 +17,88 @@ other components.
     - [DataWorkflowServices/dws](https://github.com/DataWorkflowServices/dws)
     - [HewlettPackard/lustre-csi-driver](https://github.com/HewlettPackard/lustre-csi-driver)
     - [NearNodeFlash/lustre-fs-operator](https://github.com/NearNodeFlash/lustre-fs-operator)
-    - [NearNodeFlash/nnf-mfu](https://github.com/NearNodeFlash/nnf-mfu)
-    - [NearNodeFlash/nnf-ec](https://github.com/NearNodeFlash/nnf-ec)
+    - [NearNodeFlash/nnf-mfu](https://github.com/NearNodeFlash/nnf-mfu) *(standalone repo, not a submodule)*
+    - [NearNodeFlash/nnf-ec](https://github.com/NearNodeFlash/nnf-ec) *(standalone repo, not a submodule)*
     - [NearNodeFlash/nnf-sos](https://github.com/NearNodeFlash/nnf-sos)
     - [NearNodeFlash/nnf-dm](https://github.com/NearNodeFlash/nnf-dm)
     - [NearNodeFlash/nnf-integration-test](https://github.com/NearNodeFlash/nnf-integration-test)
 
 - [NearNodeFlash/NearNodeFlash.github.io](https://github.com/NearNodeFlash/NearNodeFlash.github.io)
 
+The release order matters because of dependency relationships between repos. The `release-all.sh`
+tool enforces the following order:
+
+1. `dws`
+2. `lustre-csi-driver`
+3. `lustre-fs-operator`
+4. `nnf-mfu`
+5. `nnf-ec`
+6. `nnf-sos` (depends on dws, nnf-mfu, nnf-ec)
+7. `nnf-dm` (depends on dws, nnf-sos, nnf-mfu)
+8. `nnf-integration-test` (depends on dws, nnf-sos, nnf-dm, lustre-fs-operator)
+9. `nnf-deploy` (packages all submodules)
+10. `NearNodeFlash.github.io` (documentation, version matches nnf-deploy)
+
 ## Overview of release-all tool
 
 `release-all.sh` automates most of the steps of releasing NNF software and adds additional checks for common issues.
 
+## Prerequisites
+
+The following tools must be installed and available in your PATH:
+
+| Tool | Purpose | Notes |
+|------|---------|-------|
+| `gh` | GitHub CLI for PRs, releases | Requires `GH_TOKEN` env var (see below) |
+| `yq` | YAML processing | Must be the [Go version](https://github.com/mikefarah/yq), not the Python version |
+| `jq` | JSON processing | Used by `final-release-notes.sh` |
+| `git` | Version control | SSH access to GitHub required |
+| `make` | Build automation | Runs `make manifests`, `make generate`, etc. |
+| `perl` | Version string manipulation | Used for semver bump calculations |
+| `sed` | Text processing | |
+| `tput` | Terminal formatting | |
+
+**Environment variables:**
+
+- `GH_TOKEN` — A GitHub classic personal access token with `repo` scope. Required by `gh` and by the `release-push`, `create-pr`, `merge-pr`, and `tag-release` phases.
+
+**SSH access:**
+
+- Verify with `ssh -T git@github.com`. The release tool clones all repos via SSH.
+
 ## Assumptions
 
 - `master` or `main` branch for each repository contains **tested** software and documentation ready to be released.
-- You've installed the GitHub CLI tool, `gh`.
-    - This tool requires a GH_TOKEN environment variable containing a `repo` scope classic token.
 
 ## Steps
 
 ### Run the steps in this order
 
-> **Note:** You almost always want to use the -R option to focus the `phase` activity to a specific repo.
+> **Note:** You almost always want to use the `-R` option to focus the `phase` activity to a specific repo.
+>
+> Use `-B major|minor|patch` (default: `patch`) to control which part of the version number is bumped.
 
-0. **List Repos:** Get the ordered list of repo names to use with -R option in subsequent steps. This is referred to as `repo-list`
+0. **List Repos:** Get the ordered list of repo names to use with `-R` option in subsequent steps. This is referred to as `repo-list`
     > **Pro tip:** Keep this list in a separate window for easy viewing
+
+    ```bash
     ./release-all.sh -L
+    ```
+
+    The output will show the repos in dependency order:
+
+    ```
+    dws
+    lustre_csi_driver
+    lustre_fs_operator
+    nnf_mfu
+    nnf_ec
+    nnf_sos
+    nnf_dm
+    nnf_integration_test
+    nnf_deploy
+    nnf_doc
+    ```
 
 1. **Check Vendoring:** For each repo's master/main branch; determine whether any of them need to be re-vendored.
     > **Note:** Ensure each repo is error-free before proceeding to the next repo in `repo-list`
@@ -61,21 +116,24 @@ other components.
     ```
 
 3. **Generate Release:** For each repo in `repo-list`, proceed through the following steps in sequence before moving on to the next repo.
-    > **Note:** The next steps use the gh(1) GitHub CLI tool and require a GH_TOKEN environment variable containing a 'repo' scope classic token.
-    1. If the **Create Trial Release Branch** had no errors:
+    > **Note:** The next steps use the `gh` GitHub CLI tool and require a `GH_TOKEN` environment variable containing a `repo` scope classic token.
+
+    **Step 3a — Push the release branch.** Choose one of the following based on whether step 2 had merge conflicts:
+
+    - If the **Create Trial Release Branch** had **no errors**:
 
         ```bash
         ./release-all.sh -P release-push -R <repo>
         ```
 
-    2. If **Create Trial Release Branch** was unable to auto merge, manually fix and merge the release branch and re-run this phase on that existing branch:
+    - If **Create Trial Release Branch** was unable to auto merge, manually fix and merge the release branch, then re-run this phase on the existing branch:
 
         ```bash
-        cd workingspace/repo
+        cd workingspace/<repo>
         # Manually merge the changes from master/main to the release branch
         go mod tidy
         go mod vendor
-        git status # confirm all issues have been address
+        git status # confirm all issues have been addressed
         git add <all affected files>
         git commit -s # take the default commit message, don't bother editing it.
         ```
@@ -86,44 +144,55 @@ other components.
         USE_EXISTING_WORKAREA=1 ./release-all.sh -P release-push -R <repo>
         ```
 
-    3. Create PR for the pushed release branch:
+    **Step 3b** — Create PR for the pushed release branch:
 
-        ```bash
-        ./release-all.sh -P create-pr -R <repo>
-        ```
+    ```bash
+    ./release-all.sh -P create-pr -R <repo>
+    ```
 
-    4. Merge PR for the pushed release branch:
-    **Note: Do NOT manually merge the PR, let `release-all.sh` merge it.**
+    **Step 3c** — Merge PR for the pushed release branch:
 
-        ```bash
-        ./release-all.sh -P merge-pr -R <repo>
-        ```
+    > **Warning:** Do NOT manually merge the PR. Let `release-all.sh` merge it. If you accidentally merge manually, the `tag-release` phase may not find the expected merge commit message — use `-x force-tag=vX.Y.Z` to recover (see step 3d).
 
-    5. Tag the release:
+    ```bash
+    ./release-all.sh -P merge-pr -R <repo>
+    ```
 
-        ```bash
-        ./release-all.sh -P tag-release -R <repo>
-        ```
+    **Step 3d** — Tag the release:
+
+    > **Important:** This creates an **annotated** git tag. The CI/CD workflow (`handle_release_tag.yaml`) verifies that the tag is annotated and will reject lightweight tags. After tagging, the CI/CD workflow automatically creates the GitHub release with auto-generated release notes and attaches build artifacts (e.g., `manifests.tar` for `nnf-deploy`).
+
+    ```bash
+    ./release-all.sh -P tag-release -R <repo>
+    ```
+
+    If tagging fails because the most recent commit doesn't contain the expected merge message (e.g., due to a manual merge), use the force-tag override:
+
+    ```bash
+    ./release-all.sh -P tag-release -R <repo> -x force-tag=vX.Y.Z
+    ```
 
 ## Finalize the release notes
 
 Finalize the release by updating the `nnf-deploy` release notes to include the release notes from all submodules that were modified by this release. This also updates the release notes for any submodule that has CRDs, to include information about each version of the CRD offered by that submodule. Do this after the release steps have been completed for all repositories, including the NearNodeFlash.github.io repository.
 
+> **Note:** `final-release-notes.sh` requires `yq`, `gh`, `jq`, and `perl` to be installed, and `GH_TOKEN` to be set.
+
 1. Generate complete release notes for the specified `nnf-deploy` release for review:
 
     ```bash
     ./final-release-notes.sh -r $NNF_RELEASE
     ```
 
-2. Generate and commit the release notes to the specified `nnf-deploy` release:
+2. Review the generated notes, then commit them:
 
     ```bash
     ./final-release-notes.sh -r $NNF_RELEASE -C
     ```
 
-## Compare release manifests
+## Verify the release
 
-Compare the new NNF release manifest to a previous NNF release manifest. This can be useful for a variety of purposes. For example, this is a quick way to check for any problems in the release or to see which submodules were updated in the release.
+After all repos are released and release notes are finalized, compare the new NNF release manifest to the previous release manifest. This is a recommended verification step to confirm the release contains the expected changes and catch any problems.
 
 ```console
 ./compare-releases.sh v0.1.6 v0.1.7

mkdocs.yml

@@ -1,4 +1,4 @@
-# Release: 0.1.25
+# Release: 0.1.26
 site_name: NNF
 site_description: 'Near Node Flash'
 docs_dir: docs/