diff --git a/README.md b/README.md index ba3df41..fe21c8c 100644 --- a/README.md +++ b/README.md @@ -112,6 +112,66 @@ rm values-external.yaml Summon resolves the secrets in memory, `envsubst` renders them into a transient values file, and Helm creates the ConfigMaps/Secrets required by the Besu nodes. The temporary file is removed once the release is installed. +### Scale StatefulSet PVC storage (runbook) + +Use this runbook to grow the validator and RPC data volumes without recreating the StatefulSets. + +1. Edit your Helm values so new pods request the larger capacity and keep the updated defaults: + + ```yaml + network-nodes: + persistence: + enabled: true + storageClass: fast-ssd # cluster storage class that supports expansion + size: 200Gi # target size for every validator/RPC PVC + retention: + whenDeleted: Retain + whenScaled: Retain + ``` + +2. Roll the values into the release (reuse your existing overrides): + + ```bash + RELEASE="besu-network" + NAMESPACE="besu" + + helm upgrade --install "${RELEASE}" ./charts/network \ + --namespace "${NAMESPACE}" \ + --values values.yaml + ``` + +3. Expand the in-use PVCs with plain `kubectl` so the StatefulSets keep running while storage grows. The loop echoes success or failure for each PVC—investigate any errors (insufficient quota, permissions, driver limits) before proceeding: + + ```bash + # IMPORTANT: Set this to the same value as `network-nodes.persistence.size` from step 1. + NEW_SIZE="200Gi" + RELEASE="besu-network" + NAMESPACE="besu" + + for component in validator rpc; do + kubectl get pvc -n "${NAMESPACE}" \ + -l app.kubernetes.io/instance="${RELEASE}",app.kubernetes.io/component="${component}" \ + -o name \ + | while read -r pvc; do + if kubectl patch -n "${NAMESPACE}" "${pvc}" --type merge \ + -p "{\"spec\":{\"resources\":{\"requests\":{\"storage\":\"${NEW_SIZE}\"}}}}"; then + echo "Successfully patched ${pvc}" + else + echo "ERROR: Failed to patch ${pvc}" >&2 + fi + done + done + ``` + +4. Confirm every claim reports the larger capacity (wait for `FileSystemResizePending` to clear if your CSI driver performs an in-pod resize): + + > **Note:** The `FileSystemResizePending` status typically clears within a few minutes, but may take up to 10–15 minutes depending on your storage backend and cluster load. If the status persists longer than expected, check your CSI driver logs and node status for issues. For troubleshooting, see [Kubernetes PVC resizing documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). + + ```bash + kubectl get pvc -n "${NAMESPACE}" -l app.kubernetes.io/instance="${RELEASE}" -w + +If the StorageClass sets `allowVolumeExpansion: false`, patch it to `true` before running the loop or redeploy with a class that supports online resizing. + ### Local artefact generation with Docker Run the bootstrapper container locally to capture all artefacts before loading them into Conjur or another secret manager. diff --git a/README.tpl b/README.tpl index 040b01c..fa2e1f5 100644 --- a/README.tpl +++ b/README.tpl @@ -112,6 +112,66 @@ rm values-external.yaml Summon resolves the secrets in memory, `envsubst` renders them into a transient values file, and Helm creates the ConfigMaps/Secrets required by the Besu nodes. The temporary file is removed once the release is installed. +### Scale StatefulSet PVC storage (runbook) + +Use this runbook to grow the validator and RPC data volumes without recreating the StatefulSets. + +1. Edit your Helm values so new pods request the larger capacity and keep the updated defaults: + + ```yaml + network-nodes: + persistence: + enabled: true + storageClass: fast-ssd # cluster storage class that supports expansion + size: 200Gi # target size for every validator/RPC PVC + retention: + whenDeleted: Retain + whenScaled: Retain + ``` + +2. Roll the values into the release (reuse your existing overrides): + + ```bash + RELEASE="besu-network" + NAMESPACE="besu" + + helm upgrade --install "${RELEASE}" ./charts/network \ + --namespace "${NAMESPACE}" \ + --values values.yaml + ``` + +3. Expand the in-use PVCs with plain `kubectl` so the StatefulSets keep running while storage grows. The loop echoes success or failure for each PVC—investigate any errors (insufficient quota, permissions, driver limits) before proceeding: + + ```bash + # IMPORTANT: Set this to the same value as `network-nodes.persistence.size` from step 1. + NEW_SIZE="200Gi" + RELEASE="besu-network" + NAMESPACE="besu" + + for component in validator rpc; do + kubectl get pvc -n "${NAMESPACE}" \ + -l app.kubernetes.io/instance="${RELEASE}",app.kubernetes.io/component="${component}" \ + -o name \ + | while read -r pvc; do + if kubectl patch -n "${NAMESPACE}" "${pvc}" --type merge \ + -p "{\"spec\":{\"resources\":{\"requests\":{\"storage\":\"${NEW_SIZE}\"}}}}"; then + echo "Successfully patched ${pvc}" + else + echo "ERROR: Failed to patch ${pvc}" >&2 + fi + done + done + ``` + +4. Confirm every claim reports the larger capacity (wait for `FileSystemResizePending` to clear if your CSI driver performs an in-pod resize): + + > **Note:** The `FileSystemResizePending` status typically clears within a few minutes, but may take up to 10–15 minutes depending on your storage backend and cluster load. If the status persists longer than expected, check your CSI driver logs and node status for issues. For troubleshooting, see [Kubernetes PVC resizing documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). + + ```bash + kubectl get pvc -n "${NAMESPACE}" -l app.kubernetes.io/instance="${RELEASE}" -w + +If the StorageClass sets `allowVolumeExpansion: false`, patch it to `true` before running the loop or redeploy with a class that supports online resizing. + ### Local artefact generation with Docker Run the bootstrapper container locally to capture all artefacts before loading them into Conjur or another secret manager. diff --git a/lefthook.yml b/lefthook.yml index 793d802..8c9070b 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -4,8 +4,17 @@ pre-commit: run: bun run check:fix || true stage_fixed: true docs_cli: - run: bun run docs:cli || true + run: | + if bun run docs:cli; then + git add README.md + fi stage_fixed: true docs_helm: - run: bun run docs:helm || true + run: | + if bun run docs:helm; then + git add \ + charts/network/README.md \ + charts/network/charts/network-bootstrapper/README.md \ + charts/network/charts/network-nodes/README.md + fi stage_fixed: true