Skip to content

Commit cb507dd

Browse files
roderikroderik
authored
feat(network-nodes): add reliability controls (#27)
## Summary\n- add configurable PodDisruptionBudgets for validator and RPC StatefulSets with safeguards for min/max exclusivity\n- ship opt-in NetworkPolicy with default ingress/egress baseline for Besu pods, including secure namespace scoping and external peer egress via ipBlocks\n- expose priorityClass hooks so validator/RPC pods can preempt lower tiers\n\n## Testing\n- bun run typecheck\n- bun run check\n- bun test\n- bun run check:fix
1 parent 312a1b8 commit cb507dd

7 files changed

Lines changed: 255 additions & 1 deletion

File tree

‎.vscode/settings.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,5 +147,6 @@
147147
"workbench.editor.revealIfOpen": true,
148148
"workbench.fontAliasing": "auto",
149149
"workbench.tips.enabled": false,
150-
"workbench.tree.enableStickyScroll": true
150+
"workbench.tree.enableStickyScroll": true,
151+
"chatgpt.openOnStartup": false
151152
}

‎charts/network/charts/network-nodes/README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,13 @@ A Helm chart for Kubernetes
8888
| livenessProbe.periodSeconds | int | `10` | Frequency of liveness checks in seconds. |
8989
| livenessProbe.timeoutSeconds | int | `2` | Timeout in seconds before marking the probe as failed. |
9090
| nameOverride | string | `""` | Override for the short chart name used in resource naming. |
91+
| networkPolicy.annotations | object | `{}` | Additional annotations to add to the NetworkPolicy metadata. |
92+
| networkPolicy.egress | list | `[{"ports":[{"port":53,"protocol":"UDP"}],"to":[{"namespaceSelector":{},"podSelector":{"matchLabels":{"k8s-app":"kube-dns"}}}]},{"ports":[{"port":30303,"protocol":"TCP"}],"to":[{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"besu-statefulset"}}}]},{"ports":[{"port":30303,"protocol":"TCP"}],"to":[{"ipBlock":{"cidr":"0.0.0.0/0","except":["10.0.0.0/8","172.16.0.0/12","192.168.0.0/16"]}}]}]` | NetworkPolicy egress rules. Leave empty to deny all egress when enabled. |
93+
| networkPolicy.enabled | bool | `false` | Create a NetworkPolicy restricting Besu pod ingress and egress. |
94+
| networkPolicy.ingress | list | `[{"from":[{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"txsigner"}}},{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"erpc"}}},{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"blockscout-stack"}}},{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"graph-node"}}}],"ports":[{"port":8545,"protocol":"TCP"},{"port":8546,"protocol":"TCP"},{"port":8547,"protocol":"TCP"},{"port":9545,"protocol":"TCP"}]},{"from":[{"podSelector":{"matchLabels":{"app.kubernetes.io/name":"besu-statefulset"}}}],"ports":[{"port":30303,"protocol":"TCP"}]}]` | NetworkPolicy ingress rules. Leave empty to deny all ingress when enabled. |
95+
| networkPolicy.labels | object | `{}` | Additional labels to add to the NetworkPolicy metadata. |
96+
| networkPolicy.podSelector | object | `{}` | Optional override for the default pod selector; defaults to Besu workload labels when empty. |
97+
| networkPolicy.policyTypes | list | `[]` | Policy types enforced by the NetworkPolicy. When empty, inferred from ingress/egress rules or defaults to both. |
9198
| nodeSelector | object | `{}` | |
9299
| openShiftRoute.alternateBackends | list | `[]` | Additional backend references to balance traffic across services. |
93100
| openShiftRoute.annotations | object | `{}` | |
@@ -116,8 +123,24 @@ A Helm chart for Kubernetes
116123
| podAnnotations."prometheus.io/port" | string | `"9545"` | Container port value used by Prometheus to scrape metrics. |
117124
| podAnnotations."prometheus.io/scheme" | string | `"http"` | HTTP scheme (http or https) used for metrics scraping. |
118125
| podAnnotations."prometheus.io/scrape" | string | `"true"` | Enables Prometheus scraping of the Besu metrics endpoint. |
126+
| podDisruptionBudgets.rpc | object | `{"annotations":{},"enabled":false,"labels":{},"maxUnavailable":null,"minAvailable":1,"unhealthyPodEvictionPolicy":""}` | PodDisruptionBudget governing RPC pods. |
127+
| podDisruptionBudgets.rpc.annotations | object | `{}` | Additional annotations applied to the RPC PodDisruptionBudget. |
128+
| podDisruptionBudgets.rpc.enabled | bool | `false` | Enable the RPC PodDisruptionBudget. |
129+
| podDisruptionBudgets.rpc.labels | object | `{}` | Additional labels applied to the RPC PodDisruptionBudget. |
130+
| podDisruptionBudgets.rpc.maxUnavailable | string | `nil` | Maximum RPC pods that can be disrupted at once. Mutually exclusive with minAvailable. |
131+
| podDisruptionBudgets.rpc.minAvailable | int | `1` | Minimum RPC pods that must remain available. Mutually exclusive with maxUnavailable. |
132+
| podDisruptionBudgets.rpc.unhealthyPodEvictionPolicy | string | `""` | Optional unhealthy pod eviction policy (Default or AlwaysAllow). |
133+
| podDisruptionBudgets.validator | object | `{"annotations":{},"enabled":false,"labels":{},"maxUnavailable":null,"minAvailable":1,"unhealthyPodEvictionPolicy":""}` | PodDisruptionBudget controlling voluntary disruptions for validator pods. |
134+
| podDisruptionBudgets.validator.annotations | object | `{}` | Additional annotations applied to the validator PodDisruptionBudget. |
135+
| podDisruptionBudgets.validator.enabled | bool | `false` | Enable the validator PodDisruptionBudget. |
136+
| podDisruptionBudgets.validator.labels | object | `{}` | Additional labels applied to the validator PodDisruptionBudget. |
137+
| podDisruptionBudgets.validator.maxUnavailable | string | `nil` | Maximum validator pods that can be disrupted at once. Mutually exclusive with minAvailable. |
138+
| podDisruptionBudgets.validator.minAvailable | int | `1` | Minimum validator pods that must remain available. Mutually exclusive with maxUnavailable. |
139+
| podDisruptionBudgets.validator.unhealthyPodEvictionPolicy | string | `""` | Optional unhealthy pod eviction policy (Default or AlwaysAllow). |
119140
| podLabels | object | `{}` | |
120141
| podSecurityContext | object | `{}` | |
142+
| priorityClassNames.rpc | string | `""` | PriorityClass name assigned to RPC pods. Leave empty to inherit namespace defaults. |
143+
| priorityClassNames.validator | string | `""` | PriorityClass name assigned to validator pods. Leave empty to inherit namespace defaults. |
121144
| readinessProbe | string | `nil` | |
122145
| resources | object | `{}` | |
123146
| rpcReplicaCount | int | `2` | Number of RPC node replicas provisioned via StatefulSet. |

‎charts/network/charts/network-nodes/templates/networkpolicy.yaml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
{{- if .Values.networkPolicy.enabled }}
2+
{{- $root := . -}}
3+
{{- $np := .Values.networkPolicy | default (dict) -}}
4+
{{- $labels := get $np "labels" | default (dict) -}}
5+
{{- $annotations := get $np "annotations" | default (dict) -}}
6+
{{- $podSelector := get $np "podSelector" | default (dict) -}}
7+
{{- $hasCustomSelector := gt (len $podSelector) 0 -}}
8+
{{- $ingress := get $np "ingress" | default (list) -}}
9+
{{- $egress := get $np "egress" | default (list) -}}
10+
{{- $policyTypes := get $np "policyTypes" | default (list) -}}
11+
{{- if not $policyTypes }}
12+
{{- if gt (len $ingress) 0 }}
13+
{{- $policyTypes = append $policyTypes "Ingress" -}}
14+
{{- end }}
15+
{{- if gt (len $egress) 0 }}
16+
{{- $policyTypes = append $policyTypes "Egress" -}}
17+
{{- end }}
18+
{{- if eq (len $policyTypes) 0 }}
19+
{{- $policyTypes = list "Ingress" "Egress" -}}
20+
{{- end }}
21+
{{- end }}
22+
---
23+
apiVersion: networking.k8s.io/v1
24+
kind: NetworkPolicy
25+
metadata:
26+
name: {{ include "nodes.fullname" $root }}
27+
labels:
28+
{{- include "nodes.labels" $root | nindent 4 }}
29+
{{- with $labels }}
30+
{{- toYaml . | nindent 4 }}
31+
{{- end }}
32+
{{- with $annotations }}
33+
annotations:
34+
{{- toYaml . | nindent 4 }}
35+
{{- end }}
36+
spec:
37+
podSelector:
38+
{{- if $hasCustomSelector }}
39+
{{- toYaml $podSelector | nindent 4 }}
40+
{{- else }}
41+
matchLabels:
42+
{{- include "nodes.selectorLabels" $root | nindent 6 }}
43+
{{- end }}
44+
policyTypes:
45+
{{- toYaml $policyTypes | nindent 4 }}
46+
{{- if gt (len $ingress) 0 }}
47+
ingress:
48+
{{- toYaml $ingress | nindent 4 }}
49+
{{- end }}
50+
{{- if gt (len $egress) 0 }}
51+
egress:
52+
{{- toYaml $egress | nindent 4 }}
53+
{{- end }}
54+
{{- end }}

‎charts/network/charts/network-nodes/templates/poddisruptionbudgets.yaml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
{{- $root := . -}}
2+
{{- $pdbValues := .Values.podDisruptionBudgets | default (dict) -}}
3+
{{- $components := list
4+
(dict "name" "validator" "config" (default (dict) (get $pdbValues "validator")))
5+
(dict "name" "rpc" "config" (default (dict) (get $pdbValues "rpc")))
6+
-}}
7+
{{- range $component := $components }}
8+
{{- $cfg := $component.config -}}
9+
{{- if and $cfg (get $cfg "enabled") }}
10+
{{- $minRaw := get $cfg "minAvailable" -}}
11+
{{- $hasMin := and (hasKey $cfg "minAvailable") (ne $minRaw nil) (not (and (kindIs "string" $minRaw) (eq $minRaw ""))) -}}
12+
{{- $maxRaw := get $cfg "maxUnavailable" -}}
13+
{{- $hasMax := and (hasKey $cfg "maxUnavailable") (ne $maxRaw nil) (not (and (kindIs "string" $maxRaw) (eq $maxRaw ""))) -}}
14+
{{- $effectiveHasMin := and $hasMin (not $hasMax) -}}
15+
{{- if not (or $effectiveHasMin $hasMax) }}
16+
{{- fail (printf "podDisruptionBudgets.%s requires minAvailable or maxUnavailable" $component.name) -}}
17+
{{- end }}
18+
---
19+
apiVersion: policy/v1
20+
kind: PodDisruptionBudget
21+
metadata:
22+
name: {{ include "nodes.fullname" $root }}-{{ $component.name }}
23+
labels:
24+
{{- include "nodes.labels" $root | nindent 4 }}
25+
app.kubernetes.io/component: {{ $component.name }}
26+
{{- with (get $cfg "labels") }}
27+
{{- toYaml . | nindent 4 }}
28+
{{- end }}
29+
{{- with (get $cfg "annotations") }}
30+
annotations:
31+
{{- toYaml . | nindent 4 }}
32+
{{- end }}
33+
spec:
34+
{{- if $effectiveHasMin }}
35+
minAvailable: {{ $minRaw }}
36+
{{- end }}
37+
{{- if $hasMax }}
38+
maxUnavailable: {{ $maxRaw }}
39+
{{- end }}
40+
{{- $policy := get $cfg "unhealthyPodEvictionPolicy" -}}
41+
{{- if and $policy (ne $policy "") }}
42+
unhealthyPodEvictionPolicy: {{ $policy }}
43+
{{- end }}
44+
selector:
45+
matchLabels:
46+
{{- include "nodes.selectorLabels" $root | nindent 6 }}
47+
app.kubernetes.io/component: {{ $component.name }}
48+
{{- end }}
49+
{{- end }}

‎charts/network/charts/network-nodes/templates/statefulset-rpc.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ spec:
3131
{{- $privateKeyFilename := default "privateKey" .Values.config.privateKeyFilename }}
3232
{{- $shouldMountPersistence := $persistenceEnabled }}
3333
{{- $rpcReplicaBudget := .Values.rpcReplicaCount | int }}
34+
{{- $priorityClasses := .Values.priorityClassNames | default (dict) }}
35+
{{- $rpcPriorityClass := default "" (get $priorityClasses "rpc") }}
3436
{{- $initContainers := .Values.initContainers | default (dict) }}
3537
{{- $sharedInitContainers := get $initContainers "shared" }}
3638
{{- $rpcInitContainers := get $initContainers "rpc" }}
@@ -68,6 +70,9 @@ spec:
6870
{{- toYaml . | nindent 8 }}
6971
{{- end }}
7072
serviceAccountName: {{ include "nodes.serviceAccountName" . }}
73+
{{- if $rpcPriorityClass }}
74+
priorityClassName: {{ $rpcPriorityClass | quote }}
75+
{{- end }}
7176
{{- with .Values.podSecurityContext }}
7277
securityContext:
7378
{{- toYaml . | nindent 8 }}

‎charts/network/charts/network-nodes/templates/statefulset-validator.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,8 @@ spec:
3232
{{- $staticNodesConfigMapName := default "besu-static-nodes" (get $globalNodes "staticNodesConfigMapName") }}
3333
{{- $validatorPodPrefix := default "besu-node-validator" (get $globalNodes "podPrefix") }}
3434
{{- $validatorReplicaBudget := (include "nodes.validatorReplicaCount" . | int) }}
35+
{{- $priorityClasses := .Values.priorityClassNames | default (dict) }}
36+
{{- $validatorPriorityClass := default "" (get $priorityClasses "validator") }}
3537
{{- $initContainers := .Values.initContainers | default (dict) }}
3638
{{- $sharedInitContainers := get $initContainers "shared" }}
3739
{{- $validatorInitContainers := get $initContainers "validator" }}
@@ -69,6 +71,9 @@ spec:
6971
{{- toYaml . | nindent 8 }}
7072
{{- end }}
7173
serviceAccountName: {{ include "nodes.serviceAccountName" . }}
74+
{{- if $validatorPriorityClass }}
75+
priorityClassName: {{ $validatorPriorityClass | quote }}
76+
{{- end }}
7277
{{- with .Values.podSecurityContext }}
7378
securityContext:
7479
{{- toYaml . | nindent 8 }}

‎charts/network/charts/network-nodes/values.yaml‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,123 @@ podAnnotations:
4949
# Additional labels applied to all Besu pods.
5050
podLabels: {}
5151

52+
# PodDisruptionBudget configuration for validator and RPC workloads.
53+
podDisruptionBudgets:
54+
# -- PodDisruptionBudget controlling voluntary disruptions for validator pods.
55+
validator:
56+
# -- Enable the validator PodDisruptionBudget.
57+
enabled: false
58+
# -- Minimum validator pods that must remain available. Mutually exclusive with maxUnavailable.
59+
minAvailable: 1
60+
# -- Maximum validator pods that can be disrupted at once. Mutually exclusive with minAvailable.
61+
maxUnavailable:
62+
# -- Optional unhealthy pod eviction policy (Default or AlwaysAllow).
63+
unhealthyPodEvictionPolicy: ""
64+
# -- Additional annotations applied to the validator PodDisruptionBudget.
65+
annotations: {}
66+
# -- Additional labels applied to the validator PodDisruptionBudget.
67+
labels: {}
68+
# -- PodDisruptionBudget governing RPC pods.
69+
rpc:
70+
# -- Enable the RPC PodDisruptionBudget.
71+
enabled: false
72+
# -- Minimum RPC pods that must remain available. Mutually exclusive with maxUnavailable.
73+
minAvailable: 1
74+
# -- Maximum RPC pods that can be disrupted at once. Mutually exclusive with minAvailable.
75+
maxUnavailable:
76+
# -- Optional unhealthy pod eviction policy (Default or AlwaysAllow).
77+
unhealthyPodEvictionPolicy: ""
78+
# -- Additional annotations applied to the RPC PodDisruptionBudget.
79+
annotations: {}
80+
# -- Additional labels applied to the RPC PodDisruptionBudget.
81+
labels: {}
82+
83+
# NetworkPolicy configuration applied to Besu pods.
84+
networkPolicy:
85+
# -- Create a NetworkPolicy restricting Besu pod ingress and egress.
86+
enabled: false
87+
# -- Optional override for the default pod selector; defaults to Besu workload labels when empty.
88+
podSelector: {}
89+
# -- Policy types enforced by the NetworkPolicy. When empty, inferred from ingress/egress rules or defaults to both.
90+
policyTypes: []
91+
# -- Additional labels to add to the NetworkPolicy metadata.
92+
labels: {}
93+
# -- Additional annotations to add to the NetworkPolicy metadata.
94+
annotations: {}
95+
# -- NetworkPolicy ingress rules. Leave empty to deny all ingress when enabled.
96+
ingress:
97+
# Allow blockchain clients to reach RPC, WS, GraphQL, and metrics endpoints.
98+
- from:
99+
- podSelector:
100+
matchLabels:
101+
app.kubernetes.io/name: txsigner
102+
- podSelector:
103+
matchLabels:
104+
app.kubernetes.io/name: erpc
105+
- podSelector:
106+
matchLabels:
107+
app.kubernetes.io/name: blockscout-stack
108+
- podSelector:
109+
matchLabels:
110+
app.kubernetes.io/name: graph-node
111+
# - podSelector: {} # Uncomment to allow any pod within the namespace.
112+
ports:
113+
- protocol: TCP
114+
port: 8545
115+
- protocol: TCP
116+
port: 8546
117+
- protocol: TCP
118+
port: 8547
119+
- protocol: TCP
120+
port: 9545
121+
# Permit intra-cluster P2P traffic among Besu nodes.
122+
- from:
123+
- podSelector:
124+
matchLabels:
125+
app.kubernetes.io/name: besu-statefulset
126+
ports:
127+
- protocol: TCP
128+
port: 30303
129+
130+
# -- NetworkPolicy egress rules. Leave empty to deny all egress when enabled.
131+
egress:
132+
# Allow DNS resolution for outbound hosts.
133+
- to:
134+
- namespaceSelector: {}
135+
podSelector:
136+
matchLabels:
137+
k8s-app: kube-dns
138+
ports:
139+
- protocol: UDP
140+
port: 53
141+
# Permit Besu pods to speak to peers inside the namespace.
142+
- to:
143+
- podSelector:
144+
matchLabels:
145+
app.kubernetes.io/name: besu-statefulset
146+
ports:
147+
- protocol: TCP
148+
port: 30303
149+
# Allow outbound P2P connections to external peers (public networks).
150+
- to:
151+
- ipBlock:
152+
cidr: 0.0.0.0/0
153+
except:
154+
- 10.0.0.0/8
155+
- 172.16.0.0/12
156+
- 192.168.0.0/16
157+
# Adjust the except list to match internal cluster CIDRs to avoid bypassing other policies.
158+
ports:
159+
- protocol: TCP
160+
port: 30303
161+
162+
# PriorityClass configuration applied to Besu workloads.
163+
priorityClassNames:
164+
# -- PriorityClass name assigned to validator pods. Leave empty to inherit namespace defaults.
165+
validator: ""
166+
# -- PriorityClass name assigned to RPC pods. Leave empty to inherit namespace defaults.
167+
rpc: ""
168+
52169
# Pod-level security context shared by all containers.
53170
podSecurityContext:
54171
{}

0 commit comments

Comments
 (0)