Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,66 @@ rm values-external.yaml

Summon resolves the secrets in memory, `envsubst` renders them into a transient values file, and Helm creates the ConfigMaps/Secrets required by the Besu nodes. The temporary file is removed once the release is installed.

### Scale StatefulSet PVC storage (runbook)

Use this runbook to grow the validator and RPC data volumes without recreating the StatefulSets.

1. Edit your Helm values so new pods request the larger capacity and keep the updated defaults:

```yaml
network-nodes:
persistence:
enabled: true
storageClass: fast-ssd # cluster storage class that supports expansion
size: 200Gi # target size for every validator/RPC PVC
retention:
whenDeleted: Retain
whenScaled: Retain
```

2. Roll the values into the release (reuse your existing overrides):

```bash
RELEASE="besu-network"
NAMESPACE="besu"

helm upgrade --install "${RELEASE}" ./charts/network \
--namespace "${NAMESPACE}" \
--values values.yaml
```

3. Expand the in-use PVCs with plain `kubectl` so the StatefulSets keep running while storage grows. The loop echoes success or failure for each PVC—investigate any errors (insufficient quota, permissions, driver limits) before proceeding:

```bash
# IMPORTANT: Set this to the same value as `network-nodes.persistence.size` from step 1.
NEW_SIZE="200Gi"
Comment thread
roderik marked this conversation as resolved.
RELEASE="besu-network"
NAMESPACE="besu"

for component in validator rpc; do
kubectl get pvc -n "${NAMESPACE}" \
-l app.kubernetes.io/instance="${RELEASE}",app.kubernetes.io/component="${component}" \
-o name \
| while read -r pvc; do
if kubectl patch -n "${NAMESPACE}" "${pvc}" --type merge \
-p "{\"spec\":{\"resources\":{\"requests\":{\"storage\":\"${NEW_SIZE}\"}}}}"; then
echo "Successfully patched ${pvc}"
else
echo "ERROR: Failed to patch ${pvc}" >&2
fi
done
done
```

4. Confirm every claim reports the larger capacity (wait for `FileSystemResizePending` to clear if your CSI driver performs an in-pod resize):

> **Note:** The `FileSystemResizePending` status typically clears within a few minutes, but may take up to 10–15 minutes depending on your storage backend and cluster load. If the status persists longer than expected, check your CSI driver logs and node status for issues. For troubleshooting, see [Kubernetes PVC resizing documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims).

```bash
kubectl get pvc -n "${NAMESPACE}" -l app.kubernetes.io/instance="${RELEASE}" -w

If the StorageClass sets `allowVolumeExpansion: false`, patch it to `true` before running the loop or redeploy with a class that supports online resizing.

### Local artefact generation with Docker

Run the bootstrapper container locally to capture all artefacts before loading them into Conjur or another secret manager.
Expand Down
60 changes: 60 additions & 0 deletions README.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,66 @@ rm values-external.yaml

Summon resolves the secrets in memory, `envsubst` renders them into a transient values file, and Helm creates the ConfigMaps/Secrets required by the Besu nodes. The temporary file is removed once the release is installed.

### Scale StatefulSet PVC storage (runbook)

Use this runbook to grow the validator and RPC data volumes without recreating the StatefulSets.

1. Edit your Helm values so new pods request the larger capacity and keep the updated defaults:

```yaml
network-nodes:
persistence:
enabled: true
storageClass: fast-ssd # cluster storage class that supports expansion
size: 200Gi # target size for every validator/RPC PVC
retention:
whenDeleted: Retain
whenScaled: Retain
```

2. Roll the values into the release (reuse your existing overrides):

```bash
RELEASE="besu-network"
NAMESPACE="besu"

helm upgrade --install "${RELEASE}" ./charts/network \
--namespace "${NAMESPACE}" \
--values values.yaml
```

3. Expand the in-use PVCs with plain `kubectl` so the StatefulSets keep running while storage grows. The loop echoes success or failure for each PVC—investigate any errors (insufficient quota, permissions, driver limits) before proceeding:

```bash
# IMPORTANT: Set this to the same value as `network-nodes.persistence.size` from step 1.
NEW_SIZE="200Gi"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The NEW_SIZE variable is hardcoded here and also needs to be set in the Helm values in step 1. This can be error-prone if a user forgets to update it in both places. I suggest adding a comment to remind the user to keep these values in sync, improving the runbook's robustness.

   # IMPORTANT: Set this to the same value as `network-nodes.persistence.size` in your Helm values from step 1.
   NEW_SIZE="200Gi"

RELEASE="besu-network"
NAMESPACE="besu"

for component in validator rpc; do
kubectl get pvc -n "${NAMESPACE}" \
-l app.kubernetes.io/instance="${RELEASE}",app.kubernetes.io/component="${component}" \
-o name \
| while read -r pvc; do
if kubectl patch -n "${NAMESPACE}" "${pvc}" --type merge \
-p "{\"spec\":{\"resources\":{\"requests\":{\"storage\":\"${NEW_SIZE}\"}}}}"; then
echo "Successfully patched ${pvc}"
else
echo "ERROR: Failed to patch ${pvc}" >&2
fi
done
done
```

4. Confirm every claim reports the larger capacity (wait for `FileSystemResizePending` to clear if your CSI driver performs an in-pod resize):

> **Note:** The `FileSystemResizePending` status typically clears within a few minutes, but may take up to 10–15 minutes depending on your storage backend and cluster load. If the status persists longer than expected, check your CSI driver logs and node status for issues. For troubleshooting, see [Kubernetes PVC resizing documentation](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims).

```bash
kubectl get pvc -n "${NAMESPACE}" -l app.kubernetes.io/instance="${RELEASE}" -w

If the StorageClass sets `allowVolumeExpansion: false`, patch it to `true` before running the loop or redeploy with a class that supports online resizing.

### Local artefact generation with Docker

Run the bootstrapper container locally to capture all artefacts before loading them into Conjur or another secret manager.
Expand Down
13 changes: 11 additions & 2 deletions lefthook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,17 @@ pre-commit:
run: bun run check:fix || true
stage_fixed: true
docs_cli:
run: bun run docs:cli || true
run: |
if bun run docs:cli; then
git add README.md
fi
stage_fixed: true
docs_helm:
run: bun run docs:helm || true
run: |
if bun run docs:helm; then
git add \
charts/network/README.md \
charts/network/charts/network-bootstrapper/README.md \
charts/network/charts/network-nodes/README.md
fi
stage_fixed: true
Loading