Skip to content

Commit 8d10f62

Browse files
VietND96VietND96
authored
Docker: Add Redis external datastore support for Distributor configuration (#3137)
Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>
1 parent 1b57a66 commit 8d10f62

11 files changed

Lines changed: 163 additions & 20 deletions

‎Distributor/Dockerfile‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,6 @@ ENV SE_SESSION_REQUEST_TIMEOUT="300" \
2828
SE_SESSIONS_MAP_PORT="5556" \
2929
SE_SESSION_QUEUE_PORT="5559" \
3030
SE_DISTRIBUTOR_PORT="5553" \
31-
SE_OTEL_SERVICE_NAME="selenium-distributor"
31+
SE_OTEL_SERVICE_NAME="selenium-distributor" \
32+
SE_DISTRIBUTOR_IMPLEMENTATION="" \
33+
SE_DISTRIBUTOR_BACKEND_URL=""

‎Distributor/start-selenium-grid-distributor.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,14 @@ if [ ! -z "$SE_DISTRIBUTOR_SLOT_SELECTOR" ]; then
153153
append_se_opts "--slot-selector" "${SE_DISTRIBUTOR_SLOT_SELECTOR}"
154154
fi
155155

156+
if [ ! -z "$SE_DISTRIBUTOR_IMPLEMENTATION" ]; then
157+
append_se_opts "--distributor-implementation" "${SE_DISTRIBUTOR_IMPLEMENTATION}"
158+
fi
159+
160+
if [ ! -z "$SE_DISTRIBUTOR_BACKEND_URL" ]; then
161+
append_se_opts "--distributor-backend-url" "${SE_DISTRIBUTOR_BACKEND_URL}"
162+
fi
163+
156164
if [ ! -z "$SE_NEW_SESSION_THREAD_POOL_SIZE" ]; then
157165
append_se_opts "--newsession-threadpool-size" "${SE_NEW_SESSION_THREAD_POOL_SIZE}"
158166
fi

‎ENV_VARIABLES.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
|--------------|---------------|-------------|----------------------|
33
| SE_SCREEN_WIDTH | 1920 | Use in Node to set the screen width | |
44
| SE_SCREEN_HEIGHT | 1080 | Use in Node to set the screen height | |
5-
| SE_VIDEO_FILE_NAME | video.mp4 | Use in function video recording to set the output file name. Set `auto` for dynamic file name relying on test metadata | |
5+
| SE_VIDEO_FILE_NAME | auto | Use in function video recording to set the output file name. Set `auto` for dynamic file name relying on test metadata | |
66
| SE_FRAME_RATE | 15 | Set the frame rate for FFmpeg in video recording | |
77
| SE_CODEC | libx264 | Set the codec for FFmpeg in video recording | |
88
| SE_PRESET | -preset ultrafast | Set the preset for FFmpeg in video recording | |
@@ -105,7 +105,7 @@
105105
| SE_NODE_DOCKER_CONFIG_FILENAME | docker.toml | | |
106106
| SE_NODE_GRACEFUL_SHUTDOWN | | | |
107107
| SE_VIDEO_CONTAINER_NAME | | | |
108-
| SE_RECORD_VIDEO | true | | |
108+
| SE_RECORD_VIDEO | false | | |
109109
| SE_ENABLE_BROWSER_LEFTOVERS_CLEANUP | false | | |
110110
| SE_NODE_MAX_SESSIONS | 1 | Set the number of maximum concurrent sessions per browser Node, by default is 1 | --max-sessions |
111111
| SE_NODE_OVERRIDE_MAX_SESSIONS | false | By default is false, enable this flag for setting max session take effect in browser Node | --override-max-sessions |
@@ -168,3 +168,6 @@
168168
| SE_DYNAMIC_OVERRIDE_MAX_SESSIONS | | Enable this flag for setting max session take effect in Dynamic Node (both Docker and Kubernetes) | |
169169
| SE_FAILURE_SESSION_EVENTS | :failed,:failure,:error,:aborted | By default, a failure session event type contains ":failed", ":failure", ":error" or ":aborted" substrings that trigger the retain-on-failure sub-sequence. User can define more event types which handled in your test framework, separated by comma. | |
170170
| SE_RETAIN_ON_FAILURE | false | When true, recordings for sessions that pass are discarded immediately. Only sessions that fail (via failure events or abnormal close) retain their recordings and are queued for upload. | |
171+
| SE_DISTRIBUTOR_BACKEND_URL | | Redis URL for Distributor connection when using RedisBackedDistributor implementation, e.g. redis://redis:6379 | --distributor-backend-url |
172+
| SE_DISTRIBUTOR_IMPLEMENTATION | | Configure external datastore for Distributor. When enabled, all replicas share state through the backend (node registrations, slot reservations, health-check coordination) | --distributor-implementation |
173+
| SE_TCP_TUNNEL | false | | --tcp-tunnel |

‎charts/selenium-grid/CONFIGURATION.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,10 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
232232
| components.distributor.imagePullSecret | string | `""` | Image pull secret (see https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) |
233233
| components.distributor.newSessionThreadPoolSize | string | `nil` | Configure fixed-sized thread pool for the Distributor to create new sessions as it consumes new session requests from the queue |
234234
| components.distributor.slotSelectorStrategy | string | `""` | Full class name of non-default slot selector. This is used to select a slot in a Node once the Node has been matched |
235+
| components.distributor.externalDatastore | object | `{"backend":"redis","enabled":false,"redis":{"implementation":"org.openqa.selenium.grid.distributor.redis.RedisBackedDistributor","url":"redis://{{ $.Release.Name }}-redis-master:6379"}}` | Configure external datastore for Distributor. When enabled, all replicas share state through the backend (node registrations, slot reservations, health-check coordination), allowing zero-downtime rolling restarts. |
236+
| components.distributor.externalDatastore.enabled | bool | `false` | Enable external datastore for Distributor |
237+
| components.distributor.externalDatastore.backend | string | `"redis"` | Backend for external datastore (supported: redis) |
238+
| components.distributor.externalDatastore.redis | object | `{"implementation":"org.openqa.selenium.grid.distributor.redis.RedisBackedDistributor","url":"redis://{{ $.Release.Name }}-redis-master:6379"}` | Configure Redis backed Distributor |
235239
| components.distributor.extraEnvironmentVariables | list | `[]` | Specify extra environment variables for Distributor |
236240
| components.distributor.extraEnvFrom | list | `[]` | Specify extra environment variables from ConfigMap and Secret for Distributor |
237241
| components.distributor.affinity | object | `{}` | Specify affinity for distributor pods, this overwrites global.seleniumGrid.affinity parameter |
@@ -309,7 +313,7 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
309313
| components.sessionMap.nodeSelector | object | `{}` | Node selector for Session Map pods |
310314
| components.sessionMap.priorityClassName | string | `""` | Priority class name for Session Map pods |
311315
| components.sessionMap.externalDatastore.enabled | bool | `false` | Enable external datastore for Session Map |
312-
| components.sessionMap.externalDatastore.backend | string | `"postgresql"` | Backend for external datastore (supported: postgresql, redis). Details for each backend are described below config key |
316+
| components.sessionMap.externalDatastore.backend | string | `"redis"` | Backend for external datastore (supported: postgresql, redis). Details for each backend are described below config key |
313317
| components.sessionMap.externalDatastore.postgresql | object | `{"implementation":"org.openqa.selenium.grid.sessionmap.jdbc.JdbcBackedSessionMap","jdbcPassword":"seluser","jdbcUrl":"jdbc:postgresql://{{ $.Release.Name }}-postgresql:5432/selenium_sessions","jdbcUser":"seluser"}` | Configure database backed Session Map (https://www.selenium.dev/documentation/grid/advanced_features/external_datastore/#database-backed-session-map) |
314318
| components.sessionMap.externalDatastore.redis | object | `{"hostname":"{{ $.Release.Name }}-redis-master","implementation":"org.openqa.selenium.grid.sessionmap.redis.RedisBackedSessionMap","port":"6379","scheme":"redis"}` | Configure Redis backed Session Map (https://www.selenium.dev/documentation/grid/advanced_features/external_datastore/#redis-backed-session-map) |
315319
| components.sessionQueue.imageRegistry | string | `nil` | Registry to pull the image (this overwrites global.seleniumGrid.imageRegistry parameter) |

‎charts/selenium-grid/templates/distributor-configmap.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,17 @@ metadata:
99
labels:
1010
{{- include "seleniumGrid.commonLabels" $ | nindent 4 }}
1111
data:
12+
{{- if .Values.components.distributor.externalDatastore.enabled }}
13+
{{- $backend := .Values.components.distributor.externalDatastore.backend -}}
14+
{{- with (get .Values.components.distributor.externalDatastore $backend) }}
15+
{{- with .implementation }}
16+
SE_DISTRIBUTOR_IMPLEMENTATION: {{ tpl . $ | quote }}
17+
{{- end }}
18+
{{- with .url }}
19+
SE_DISTRIBUTOR_BACKEND_URL: {{ tpl . $ | quote }}
20+
{{- end }}
21+
{{- end }}
22+
{{- end }}
1223
{{- $fileProceeded := list -}}
1324
{{- range $path, $_ := .Files.Glob $.Values.distributorConfigMap.extraScriptsImportFrom }}
1425
{{- $fileName := base $path -}}

‎charts/selenium-grid/values.yaml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -566,6 +566,18 @@ components:
566566
newSessionThreadPoolSize:
567567
# -- Full class name of non-default slot selector. This is used to select a slot in a Node once the Node has been matched
568568
slotSelectorStrategy: ""
569+
# -- Configure external datastore for Distributor. When enabled, all replicas share state
570+
# through the backend (node registrations, slot reservations, health-check coordination),
571+
# allowing zero-downtime rolling restarts.
572+
externalDatastore:
573+
# -- Enable external datastore for Distributor
574+
enabled: false
575+
# -- Backend for external datastore (supported: redis)
576+
backend: redis
577+
# -- Configure Redis backed Distributor
578+
redis:
579+
implementation: "org.openqa.selenium.grid.distributor.redis.RedisBackedDistributor"
580+
url: "redis://{{ $.Release.Name }}-redis-master:6379"
569581
# -- Specify extra environment variables for Distributor
570582
extraEnvironmentVariables: []
571583
# -- Specify extra environment variables from ConfigMap and Secret for Distributor
@@ -771,7 +783,7 @@ components:
771783
# -- Enable external datastore for Session Map
772784
enabled: false
773785
# -- Backend for external datastore (supported: postgresql, redis). Details for each backend are described below config key
774-
backend: postgresql
786+
backend: redis
775787
# -- Configure database backed Session Map (https://www.selenium.dev/documentation/grid/advanced_features/external_datastore/#database-backed-session-map)
776788
postgresql:
777789
implementation: "org.openqa.selenium.grid.sessionmap.jdbc.JdbcBackedSessionMap"

docker-compose-v3-full-grid-external-datastore.yml renamed to docker-compose-v3-full-grid-external-postgres-backed.yml

Lines changed: 2 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# To execute this docker compose yml file use `docker compose -f docker-compose-v3-full-grid.yml up`
1+
# To execute this docker compose yml file use `docker compose -f docker-compose-v3-full-grid-external-postgres-backed.yml.yml up`
22
# Add the `-d` flag at the end for detached execution
3-
# To stop the execution, hit Ctrl+C, and then `docker compose -f docker-compose-v3-full-grid.yml down`
3+
# To stop the execution, hit Ctrl+C, and then `docker compose -f docker-compose-v3-full-grid-external-postgres-backed.yml.yml down`
44
services:
55
selenium-event-bus:
66
image: selenium/event-bus:4.43.0-20260404
@@ -24,11 +24,6 @@ services:
2424
- SE_SESSIONS_MAP_EXTERNAL_JDBC_URL=jdbc:postgresql://postgresql:5432/selenium_sessions
2525
- SE_SESSIONS_MAP_EXTERNAL_JDBC_USER=seluser
2626
- SE_SESSIONS_MAP_EXTERNAL_JDBC_PASSWORD=seluser
27-
# Uncomment the following lines to use Redis as the external datastore
28-
# - SE_SESSIONS_MAP_EXTERNAL_SCHEME=redis
29-
# - SE_SESSIONS_MAP_EXTERNAL_IMPLEMENTATION=org.openqa.selenium.grid.sessionmap.redis.RedisBackedSessionMap
30-
# - SE_SESSIONS_MAP_EXTERNAL_HOSTNAME=redis
31-
# - SE_SESSIONS_MAP_EXTERNAL_PORT=6379
3227

3328
postgresql:
3429
image: postgres:latest
@@ -42,12 +37,6 @@ services:
4237
volumes:
4338
- ./Sessions/init.sql:/docker-entrypoint-initdb.d/init.sql
4439

45-
redis:
46-
image: redis:latest
47-
restart: always
48-
ports:
49-
- "6379:6379"
50-
5140
selenium-session-queue:
5241
image: selenium/session-queue:4.43.0-20260404
5342
container_name: selenium-session-queue

‎docker-compose-v3-full-grid-external-redis-backed.yml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# To execute this docker compose yml file use `docker compose -f docker-compose-v3-full-grid-external-redis-backed.yml up`
2+
# Add the `-d` flag at the end for detached execution
3+
# To stop the execution, hit Ctrl+C, and then `docker compose -f docker-compose-v3-full-grid-external-redis-backed.yml down`
4+
services:
5+
selenium-event-bus:
6+
image: selenium/event-bus:4.43.0-20260404
7+
container_name: selenium-event-bus
8+
ports:
9+
- "4442:4442"
10+
- "4443:4443"
11+
- "5557:5557"
12+
13+
selenium-sessions:
14+
image: selenium/sessions:4.43.0-20260404
15+
container_name: selenium-sessions
16+
ports:
17+
- "5556:5556"
18+
depends_on:
19+
- selenium-event-bus
20+
environment:
21+
- SE_EVENT_BUS_HOST=selenium-event-bus
22+
- SE_SESSIONS_MAP_EXTERNAL_DATASTORE=true
23+
- SE_SESSIONS_MAP_EXTERNAL_SCHEME=redis
24+
- SE_SESSIONS_MAP_EXTERNAL_IMPLEMENTATION=org.openqa.selenium.grid.sessionmap.redis.RedisBackedSessionMap
25+
- SE_SESSIONS_MAP_EXTERNAL_HOSTNAME=redis
26+
- SE_SESSIONS_MAP_EXTERNAL_PORT=6379
27+
28+
redis:
29+
image: redis:latest
30+
restart: always
31+
32+
selenium-session-queue:
33+
image: selenium/session-queue:4.43.0-20260404
34+
container_name: selenium-session-queue
35+
ports:
36+
- "5559:5559"
37+
38+
selenium-distributor:
39+
image: selenium/distributor:4.43.0-20260404
40+
container_name: selenium-distributor
41+
ports:
42+
- "5553:5553"
43+
depends_on:
44+
- selenium-event-bus
45+
- selenium-sessions
46+
- selenium-session-queue
47+
environment:
48+
- SE_EVENT_BUS_HOST=selenium-event-bus
49+
- SE_SESSIONS_MAP_HOST=selenium-sessions
50+
- SE_SESSION_QUEUE_HOST=selenium-session-queue
51+
- SE_DISTRIBUTOR_IMPLEMENTATION=org.openqa.selenium.grid.distributor.redis.RedisBackedDistributor
52+
- SE_DISTRIBUTOR_BACKEND_URL=redis://redis:6379
53+
54+
selenium-router:
55+
image: selenium/router:4.43.0-20260404
56+
container_name: selenium-router
57+
ports:
58+
- "4444:4444"
59+
depends_on:
60+
- selenium-distributor
61+
- selenium-sessions
62+
- selenium-session-queue
63+
environment:
64+
- SE_DISTRIBUTOR_HOST=selenium-distributor
65+
- SE_SESSIONS_MAP_HOST=selenium-sessions
66+
- SE_SESSION_QUEUE_HOST=selenium-session-queue
67+
68+
chrome:
69+
image: selenium/node-chrome:4.43.0-20260404
70+
platform: linux/amd64
71+
shm_size: 2gb
72+
depends_on:
73+
- selenium-event-bus
74+
environment:
75+
- SE_EVENT_BUS_HOST=selenium-event-bus
76+
77+
edge:
78+
image: selenium/node-edge:4.43.0-20260404
79+
platform: linux/amd64
80+
shm_size: 2gb
81+
depends_on:
82+
- selenium-event-bus
83+
environment:
84+
- SE_EVENT_BUS_HOST=selenium-event-bus
85+
86+
firefox:
87+
image: selenium/node-firefox:4.43.0-20260404
88+
shm_size: 2gb
89+
depends_on:
90+
- selenium-event-bus
91+
environment:
92+
- SE_EVENT_BUS_HOST=selenium-event-bus

‎scripts/generate_list_env_vars/description.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -554,3 +554,15 @@
554554
Only sessions that fail (via failure events or abnormal close) retain their recordings
555555
and are queued for upload.
556556
cli: ''
557+
- name: SE_DISTRIBUTOR_BACKEND_URL
558+
description: Redis URL for Distributor connection when using RedisBackedDistributor
559+
implementation, e.g. redis://redis:6379
560+
cli: --distributor-backend-url
561+
- name: SE_DISTRIBUTOR_IMPLEMENTATION
562+
description: Configure external datastore for Distributor. When enabled, all replicas
563+
share state through the backend (node registrations, slot reservations, health-check
564+
coordination)
565+
cli: --distributor-implementation
566+
- name: SE_TCP_TUNNEL
567+
description: ''
568+
cli: --tcp-tunnel

‎scripts/generate_list_env_vars/value.yaml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,12 @@
1616
default: libx264
1717
- name: SE_DISABLE_UI
1818
default: ''
19+
- name: SE_DISTRIBUTOR_BACKEND_URL
20+
default: ''
1921
- name: SE_DISTRIBUTOR_HOST
2022
default: ''
23+
- name: SE_DISTRIBUTOR_IMPLEMENTATION
24+
default: ''
2125
- name: SE_DISTRIBUTOR_PORT
2226
default: '5553'
2327
- name: SE_DISTRIBUTOR_SLOT_SELECTOR
@@ -197,7 +201,7 @@
197201
- name: SE_RECORD_AUDIO
198202
default: 'false'
199203
- name: SE_RECORD_VIDEO
200-
default: 'true'
204+
default: 'false'
201205
- name: SE_REGISTRATION_SECRET
202206
default: ''
203207
- name: SE_REJECT_UNSUPPORTED_CAPS
@@ -276,6 +280,8 @@
276280
default: '5'
277281
- name: SE_SUPERVISORD_UNIX_SERVER_PASSWORD
278282
default: secret
283+
- name: SE_TCP_TUNNEL
284+
default: 'false'
279285
- name: SE_UPDATE_CHROME_COMPONENTS
280286
default: ''
281287
- name: SE_UPLOAD_COMMAND
@@ -297,7 +303,7 @@
297303
- name: SE_VIDEO_EVENT_DRIVEN
298304
default: 'true'
299305
- name: SE_VIDEO_FILE_NAME
300-
default: video.mp4
306+
default: auto
301307
- name: SE_VIDEO_FILE_NAME_SUFFIX
302308
default: 'true'
303309
- name: SE_VIDEO_FILE_NAME_TRIM_REGEX

0 commit comments

Comments
 (0)