Skip to content

Commit 9bd2be1

Browse files
authored
update valkey to use hostname (agent-substrate#266)
Fixes agent-substrate#225 - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR Tested local setup: ``` ❯ kubectl exec -n ate-system valkey-cluster-0 -- cat /tmp/valkey.conf | grep cluster-announce cluster-announce-hostname valkey-cluster-0.valkey-cluster-service.ate-system.svc cluster-announce-tls-port 6379 cluster-announce-bus-port 16379 ``` Separate from this, we should decrease the number of clusters, because right now all of these get created: ``` ❯ kubectl exec -n ate-system valkey-cluster-0 -- valkey-cli --tls --cacert /etc/valkey-ca/ca.crt --cert /run/servicedns.podcert.ate.dev/credential-bundle.pem --key /run/servicedns.podcert.ate.dev/credential-bundle.pem cluster nodes d6ca3f519ddcf774bc8df68f7b7ab6c82cc35066 10.244.0.25:6379@16379,valkey-cluster-2.valkey-cluster-service.ate-system.svc master - 0 1781721223000 3 connected 10923-16383 6f66ad32594c1a5dc550729434996eb129112a1f 10.244.0.27:6379@16379,valkey-cluster-3.valkey-cluster-service.ate-system.svc slave d6ca3f519ddcf774bc8df68f7b7ab6c82cc35066 0 1781721224614 3 connected c72d9e9bd165b0f800a135fbda1d8184e8111319 10.244.0.28:6379@16379,valkey-cluster-4.valkey-cluster-service.ate-system.svc slave 1c37cdd7fdcc0a261614a6e3936cc0043fe63d35 0 1781721223594 1 connected 54690be32b6647d43e5e1171e6db92ddf9515ad7 10.244.0.20:6379@16379,valkey-cluster-1.valkey-cluster-service.ate-system.svc master - 0 1781721224614 2 connected 5461-10922 6d862697f8af8750a5280409d1eb2bce9f3f8e0a 10.244.0.26:6379@16379,valkey-cluster-5.valkey-cluster-service.ate-system.svc slave 54690be32b6647d43e5e1171e6db92ddf9515ad7 0 1781721223594 2 connected 1c37cdd7fdcc0a261614a6e3936cc0043fe63d35 10.244.0.22:6379@16379,valkey-cluster-0.valkey-cluster-service.ate-system.svc myself,master - 0 0 1 connected 0-5460 ``` agent-substrate#267 is a quick fix to decrease the pain of setting up the cluster + tearing it down. This would fix the stale IP issue. Signed-off-by: npolshakova <nina.polshakova@solo.io>
1 parent f45e42e commit 9bd2be1

1 file changed

Lines changed: 18 additions & 7 deletions

File tree

manifests/ate-install/valkey.yaml

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,20 @@ spec:
8888
containers:
8989
- name: valkey
9090
image: valkey/valkey:8.0
91-
command: ["valkey-server", "/etc/valkey/valkey.conf"]
91+
command:
92+
- /bin/sh
93+
- -c
94+
- |
95+
set -e
96+
POD_DNS="${HOSTNAME}.valkey-cluster-service.ate-system.svc"
97+
cp /etc/valkey/valkey.conf /tmp/valkey.conf
98+
{
99+
echo "cluster-announce-hostname ${POD_DNS}"
100+
echo "cluster-announce-tls-port 6379"
101+
echo "cluster-announce-bus-port 16379"
102+
echo "cluster-preferred-endpoint-type hostname"
103+
} >> /tmp/valkey.conf
104+
exec valkey-server /tmp/valkey.conf
92105
ports:
93106
- name: valkey
94107
containerPort: 6379
@@ -166,11 +179,10 @@ spec:
166179
done
167180
done
168181
169-
echo "All pods resolved. Getting IPs..."
170-
POD_IPS=""
182+
echo "All pods resolved. Building stable node list..."
183+
VALKEY_NODES=""
171184
for i in 0 1 2 3 4 5; do
172-
ip=$(getent hosts valkey-cluster-${i}.valkey-cluster-service.ate-system.svc | awk '{print $1}')
173-
POD_IPS="${POD_IPS} ${ip}:6379"
185+
VALKEY_NODES="${VALKEY_NODES} valkey-cluster-${i}.valkey-cluster-service.ate-system.svc:6379"
174186
done
175187
176188
echo "Checking if Valkey cluster is already initialized..."
@@ -187,7 +199,7 @@ spec:
187199
--cacert /etc/valkey-ca/ca.crt \
188200
--cert /run/servicedns.podcert.ate.dev/credential-bundle.pem \
189201
--key /run/servicedns.podcert.ate.dev/credential-bundle.pem \
190-
--cluster create ${POD_IPS} \
202+
--cluster create ${VALKEY_NODES} \
191203
--cluster-replicas 1 \
192204
--cluster-yes
193205
echo "Cluster initialization complete!"
@@ -210,4 +222,3 @@ spec:
210222
items:
211223
- key: ca.crt
212224
path: ca.crt
213-

0 commit comments

Comments
 (0)