feat(helm): Use Valkey

[kiblik]

Fix #13323

#13331 was replaced only in docker-compose. This PR replaces it in the HELM chart as well.
Official Valkey HELM charts would require too many customisations; we needed something more developed. https://artifacthub.io/packages/helm/cloudpirates-valkey/valkey supported many fields that are compatible with the previous Bitnami stack.

There are multiple places that are mentioned in release notes.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[Maffooch]

Hi @kiblik the valkey migration is slated for release in the upcoming minor release on November 3rd. The docker compose PR looks ready to roll after your feedback was incorporated. Is there anything left to be implemented in this PR for the helm chart?

[kiblik]

So, my original idea was to replace Bitnami Redis with the official Valkey Helm chart. Unfortunately, this is not fully possible because authN issue.
The official Valkey Helm chart does not support the approach: "This is a password which I generated/prepared for you. Use it". Valkey needs the users.acl file with the full definition of users.
This issue is partially solved in the open PR, but still not fully support "This is a prepared password. Use it."
So we might

• use Valkey chart from Bitnami. But charts and/or Docker images would have to be mirrored (not a good and maintainable solution)
• use CloudPirates chart (comes from the original Bitnami definition, but they keep it nicely maintained)
• use deployment without authN (we might depend on network isolation using NetworkPolicy)
• ???

[kiblik]

I hope migration will be easier, and I did not have enough time to come back to it until now.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[kiblik]

@fernandezcuesta, couple more adjustments were needed (replace template with include or missing . in template), but the base was pretty good. Btw, I also added redisPort (just in case).

I do not have access to rediss or sentinel or any other external instance so I have not been able to test all scenarios. But helm template ... --show-only templates/configmap.yaml | grep 'DD_CELERY_BROKER' was adjusting values based on my expectations. So I hope we are good.

@fernandezcuesta, feel free to have one more look (resolve open conversations).

@Maffooch, @mtesauro, @rossops, this PR is ready for review.

[kiblik]

So, my original idea was to replace Bitnami Redis with the official Valkey Helm chart. Unfortunately, this is not fully possible because authN issue. The official Valkey Helm chart does not support the approach: "This is a password which I generated/prepared for you. Use it". Valkey needs the users.acl file with the full definition of users. This issue is partially solved in the open PR, but still not fully support "This is a prepared password. Use it." So we might

Hi, I'm one of the Valkey Helm chart maintainers. We have implemented both inline and secret-based passwords in the chart. Note that it currently only supports standalone deployments, with replicas coming soon.

@sgissi, thank you for your input. We decided to go with CloudPirates for a smoother transition.

[valentijnscholten]

Thank you @kiblik for looking in to this, great job. For the docker compose setup we have chosen not to rename anything to keep it simple. But that setup only uses a volume named "xxx_redis". For the helm chart there are more value and options at play so it makes sense to rename them.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[mtesauro]

Approved

Nice job! 🚀