Skip to content

fix: Add new monitors worker for temporal #318

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gtoonstra merged 1 commit into from
May 5, 2026
Merged

fix: Add new monitors worker for temporal #318

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/datafold-manager/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: datafold-manager
description: Helm chart for Datafold Operator
type: application
version: 0.1.95
version: 0.1.96
appVersion: "1.0.0"
icon: https://www.datafold.com/logo.png

Expand Down
171 changes: 86 additions & 85 deletions charts/datafold-manager/templates/crds.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2249,9 +2249,9 @@ spec:
type: object
type: object
type: object
worker-realtime:
description: WorkerRealtime enables the realtime temporal worker
(interactive, api queues)
worker-monitors:
description: WorkerMonitors enables the monitors temporal worker
(monitors queue)
properties:
command:
description: Command is the container command (e.g. ["./manage.py",
Expand Down Expand Up @@ -2529,83 +2529,9 @@ spec:
format: int64
type: integer
type: object
worker-singletons:
description: WorkerSingletons enables the singletons worker
properties:
enabled:
description: Enabled specifies whether this component should
be installed
type: boolean
rawValues:
description: |-
RawValues contains arbitrary Helm values to pass directly to this component's subchart
These values will be merged with the component's default values and can override any
configuration option supported by the component's Helm chart
x-kubernetes-preserve-unknown-fields: true
replicas:
description: Replicas specifies the number of replicas for
this component
format: int32
type: integer
resources:
description: Resources specifies the resource requirements
for this component
properties:
limits:
description: Limits specifies the maximum resources that
can be used
properties:
cpu:
description: CPU specifies the maximum CPU resources
type: string
ephemeralStorage:
description: EphemeralStorage specifies the maximum
ephemeral storage (e.g. "10Gi")
type: string
hugepages1Gi:
description: Hugepages1Gi specifies the maximum 1Gi
hugepages (e.g. "1Gi"). Requests must equal limits
for hugepages.
type: string
hugepages2Mi:
description: Hugepages2Mi specifies the maximum 2Mi
hugepages (e.g. "128Mi"). Requests must equal limits
for hugepages.
type: string
memory:
description: Memory specifies the maximum memory resources
type: string
type: object
requests:
description: Requests specifies the minimum resources
that should be allocated
properties:
cpu:
description: CPU specifies the minimum CPU resources
type: string
ephemeralStorage:
description: EphemeralStorage specifies the minimum
ephemeral storage (e.g. "10Gi")
type: string
hugepages1Gi:
description: Hugepages1Gi specifies the minimum 1Gi
hugepages (e.g. "1Gi"). Must equal limits when both
are set.
type: string
hugepages2Mi:
description: Hugepages2Mi specifies the minimum 2Mi
hugepages (e.g. "128Mi"). Must equal limits when
both are set.
type: string
memory:
description: Memory specifies the minimum memory resources
type: string
type: object
type: object
type: object
worker-storage:
description: WorkerStorage enables the storage temporal worker
(localstorage, replication queues)
worker-realtime:
description: WorkerRealtime enables the realtime temporal worker
(interactive, api queues)
properties:
command:
description: Command is the container command (e.g. ["./manage.py",
Expand Down Expand Up @@ -2883,9 +2809,83 @@ spec:
format: int64
type: integer
type: object
worker-storage-high:
description: WorkerStorageHigh enables the high-capacity storage
temporal worker (localstorage-large queue)
worker-singletons:
description: WorkerSingletons enables the singletons worker
properties:
enabled:
description: Enabled specifies whether this component should
be installed
type: boolean
rawValues:
description: |-
RawValues contains arbitrary Helm values to pass directly to this component's subchart
These values will be merged with the component's default values and can override any
configuration option supported by the component's Helm chart
x-kubernetes-preserve-unknown-fields: true
replicas:
description: Replicas specifies the number of replicas for
this component
format: int32
type: integer
resources:
description: Resources specifies the resource requirements
for this component
properties:
limits:
description: Limits specifies the maximum resources that
can be used
properties:
cpu:
description: CPU specifies the maximum CPU resources
type: string
ephemeralStorage:
description: EphemeralStorage specifies the maximum
ephemeral storage (e.g. "10Gi")
type: string
hugepages1Gi:
description: Hugepages1Gi specifies the maximum 1Gi
hugepages (e.g. "1Gi"). Requests must equal limits
for hugepages.
type: string
hugepages2Mi:
description: Hugepages2Mi specifies the maximum 2Mi
hugepages (e.g. "128Mi"). Requests must equal limits
for hugepages.
type: string
memory:
description: Memory specifies the maximum memory resources
type: string
type: object
requests:
description: Requests specifies the minimum resources
that should be allocated
properties:
cpu:
description: CPU specifies the minimum CPU resources
type: string
ephemeralStorage:
description: EphemeralStorage specifies the minimum
ephemeral storage (e.g. "10Gi")
type: string
hugepages1Gi:
description: Hugepages1Gi specifies the minimum 1Gi
hugepages (e.g. "1Gi"). Must equal limits when both
are set.
type: string
hugepages2Mi:
description: Hugepages2Mi specifies the minimum 2Mi
hugepages (e.g. "128Mi"). Must equal limits when
both are set.
type: string
memory:
description: Memory specifies the minimum memory resources
type: string
type: object
type: object
type: object
worker-storage:
description: WorkerStorage enables the storage temporal worker
(localstorage, replication queues)
properties:
command:
description: Command is the container command (e.g. ["./manage.py",
Expand Down Expand Up @@ -3163,8 +3163,9 @@ spec:
format: int64
type: integer
type: object
worker-temporal:
description: WorkerTemporal enables the base temporal worker
worker-storage-high:
description: WorkerStorageHigh enables the high-capacity storage
temporal worker (localstorage-large queue)
properties:
command:
description: Command is the container command (e.g. ["./manage.py",
Expand Down
2 changes: 1 addition & 1 deletion charts/datafold-manager/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ operator:
# Operator image configuration
image:
repository: us-docker.pkg.dev/datadiff-mm/datafold/datafold-operator
tag: "1.1.65"
tag: "1.1.66"
pullPolicy: Always

# Operator deployment configuration
Expand Down
11 changes: 6 additions & 5 deletions charts/datafold/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: datafold
description: Helm chart package to deploy Datafold on kubernetes.
type: application
version: 0.10.79
version: 0.10.80
appVersion: "1.0.0"
icon: https://www.datafold.com/logo.png

Expand All @@ -24,10 +24,6 @@ dependencies:
version: "*.*.*"
repository: file://charts/dfshell
condition: dfshell.install
- name: worker-temporal
version: "*.*.*"
repository: file://charts/worker-temporal
condition: worker-temporal.install
- name: worker-temporal
version: "*.*.*"
repository: file://charts/worker-temporal
Expand Down Expand Up @@ -58,6 +54,11 @@ dependencies:
repository: file://charts/worker-temporal
condition: worker-storage-high.install
alias: worker-storage-high
- name: worker-temporal
version: "*.*.*"
repository: file://charts/worker-temporal
condition: worker-monitors.install
alias: worker-monitors
- name: operator
version: "*.*.*"
repository: file://charts/operator
Expand Down
30 changes: 16 additions & 14 deletions charts/datafold/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,20 +230,6 @@ toplevel:
labels: {}
annotations: {}

worker-temporal:
install: false
replicaCount: 0
terminationGracePeriodSeconds: "30"
temporal:
taskQueues: "all"
workerPoolType: "thread"
resources:
limits:
memory: 4Gi
requests:
cpu: 500m
memory: 4Gi

worker-io:
install: true
replicaCount: 0
Expand Down Expand Up @@ -334,6 +320,22 @@ worker-storage-high:
cpu: 100m
memory: 30Gi

worker-monitors:
install: true
replicaCount: 0
terminationGracePeriodSeconds: "30"
temporal:
taskQueues: "monitors"
workerPoolType: "process"
keda:
minReplicas: 0
resources:
limits:
memory: 15Gi
requests:
cpu: 200m
memory: 15Gi

worker:
install: true
replicaCount: 1
Expand Down
Loading