docker compose: switch to Valkey as message broker#13331
docker compose: switch to Valkey as message broker#13331mtesauro merged 17 commits intoDefectDojo:devfrom
Conversation
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
1 similar comment
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
For the record, I'm all for this approach. Keeping an 👁️ on this for it to leave Draft status 😉 |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
Hello @valentijnscholten This migration fixes the Redis latest vulnerability (CVE-2025-49844) that was fixed in v8.2.2? |
|
Yes it does: https://github.com/valkey-io/valkey/releases/tag/7.2.11, but we'll be updating redis to 7.2.12 before we're moving to valkey as we still have some testing/adjustments to do to the helm chart. |
|
Looks there's some contradicting info floating around as according to Redis themselves the CVE is fixes in 7.2.11, which we are already running: https://github.com/redis/redis/releases/tag/7.2.11 |
This was my understanding aka DefectDojo was not vulnerable when this was reported due to the version we had in the lastest version. So, keep those DefectDojo instances up to the latest, greatest version 👍 |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Conflicts have been resolved. A maintainer will review the pull request shortly. |
Updated documentation for version 2.52 to reflect the transition from Redis to Valkey as the message broker, including UI fixes and deduplication improvements.
|
This pull request includes docker-compose.yml with hardcoded default secrets (DD_SECRET_KEY and DD_CREDENTIAL_AES_256_KEY); while intended for development, leaving them in a file usable in production risks session compromise and credential decryption if not overridden. Consider removing defaults, loading secrets from environment or a secrets manager, or ensuring the compose file is strictly gated to non-production use.
Hardcoded Default Secrets in
|
| Vulnerability | Hardcoded Default Secrets |
|---|---|
| Description | The docker-compose.yml file contains hardcoded default values for DD_SECRET_KEY and DD_CREDENTIAL_AES_256_KEY. While these are intended as defaults for development environments, their presence in a file that could be used for production deployments without explicit overrides poses a risk. If these values are not overridden, an attacker who knows these default keys could compromise session integrity and decrypt sensitive credentials. |
django-DefectDojo/docker-compose.yml
Lines 53 to 54 in 5440c51
All finding details can be found in the DryRun Security Dashboard.
* docker compose: switch to valkey as message broker * docker compose: switch to valkey as message broker * docker compose: switch to valkey as message broker * docker compose: switch to valkey as message broker * docker compose: switch to valkey as message broker * Update 2.52.md * Update docs/content/en/open_source/upgrading/2.52.md Co-authored-by: kiblik <5609770+kiblik@users.noreply.github.com> * Update 2.52.md * Update docs/content/en/open_source/upgrading/2.52.md * Revise 2.52 upgrade notes for Valkey integration Updated documentation for version 2.52 to reflect the transition from Redis to Valkey as the message broker, including UI fixes and deduplication improvements. --------- Co-authored-by: kiblik <5609770+kiblik@users.noreply.github.com>
Fixes #13323
Since the license change at Redis the fork ValKey has become widely popular and is backed by industry giants such as AWS. AWS is advising to use ValKey over Redis and is using lower prices for ValKey compared to Redis.
This PR migrates the docker compose setup to use Valkey instead of Redis.
The simplest approach is chosen:
Justification
Since Valkey 7.2 is a fork of Redis 7.2 it is a drop-in replacement. I've tested the by generating a large backlog of celery task before making the changes in this PR. Shuttting down, applying the changes and bringin Defect Dojo up resulted in celeryworkers resuming work and processing the backlog.
I've checked the Dockerfiles for both Redis and Valkey. They show that they're laregely the same:
/datato store their dataThe Valkey docs mention alternative migration paths such as taking snapshots and restoring these:
The re-use of the existing volume is much simpler and fits with the statement that Valkey 7.2 is compatible with Redis 7.2
In theory we could go to Valkey 8 or 9, but somehow it feels like the safer option to stick with 7.2 for now.
Client libraries
I tried migrating from the
redisPython library to thevalkeyPython library, but ran into some issues with Celery. Research shows it's recommended to stick to theredislibrary unitl all other frameworks such as Celery have adopted first-class support of the Valkey platform.Volume names still contains
redisBy re-using the existing docker volume, we are stuck with the name
defectdojo_redis. We can't rename volumes, so we just have to live with this. Or.....EDIT: We've chosen to retain the volume and accept for now that it is named
defectdojo_redis1) Its possible to change the name of the volume todefectdojo_valkeyif and only if users perform some manual steps during the upgrade. This would look something like:~~ ~~`docker compose down`~~ ~~`docker volume create XXXX_defectdojo_valkey`~~ ~~`docker run --rm -u 0:0 -v oldvol:/from -v XXXX_defectdojo_valkey:/to alpine sh -c 'tar -C /from -cpf - . | tar -C /to -xpf -'`~~ ~~`docker compose up -d`~~ ~~`docker volume rm XXXX_defectdojo_redis`~~ ~~The XXX is the prefix given to volume names by docker compose, usually based on the folder Defect Dojo is in.In theory we could create a script that reads the exact volume name and have amigrate-redis-to-valkey.shscript that does everything transparently.2) That script could (in theory) also be run by theinitializer. Something like this could be added to the entrypoint:~~ ~~# Optional one-time migration from old Redis volume (/redis-data) to new Valkey volume (/valkey-data)~~ ~~# Runs before DD_INITIALIZE check so it executes even if initialization is skipped~~ ~~if [ "${DD_MIGRATE_REDIS_VOLUME:-1}" != "0" ] && [ -d /redis-data ] && [ -d /valkey-data ]; then~~ ~~ if [ ! -f /valkey-data/.dojo_redis_migrated ] \~~ ~~ && [ -z "$(ls -A /valkey-data 2>/dev/null || true)" ] \~~ ~~ && [ -n "$(ls -A /redis-data 2>/dev/null || true)" ]; then~~ ~~ echo "Migrating data from old Redis volume (/redis-data) to new Valkey volume (/valkey-data)..."~~ ~~ tar -C /redis-data -cpf - . | tar -C /valkey-data -xpf -~~ ~~ touch /valkey-data/.dojo_redis_migrated~~ ~~ else~~ ~~ echo "Skip migration: destination not empty, source empty, or already migrated"~~ ~~ fi~~ ~~fi~~ ~~3) Or we could add a one-time init service to the docker-compose file. Something like:```services:~~ valkey-init-migrate:~~
~~ image: alpine:3.19~~
~~ command: >~~
~~ sh -c 'set -eu;~~
~~ if [ -z "$(ls -A /to 2>/dev/null)" ] && [ -n "$(ls -A /from 2>/dev/null)" ]; then~~
~~ tar -C /from -cpf - . | tar -C /to -xpf -;~~
~~ else~~
~~ echo "Skip copy: /to not empty or /from empty";~~
~~ fi'~~
~~ volumes:~~
~~ - oldredisdata:/from:ro~~
~~ - valkeydata:/to~~
~~ restart: "no"~~
~~ valkey:~~
~~ image: valkey/valkey:7.2-alpine~~
~~ depends_on:~~
~~ valkey-init-migrate:~~
~~ condition: service_completed_successfully~~
~~ volumes:~~
~~ - valkeydata:/data~~
~~ # command: ["valkey-server","--appendonly","yes"] # if you used AOF~~
volumes:~~ oldredisdata:~~
~~ external: true # existing Redis volume name~~
~~ name: oldredisdata~~
~~ valkeydata:~~
~~ # new volume Compose will create (your new name)~~
```I say in theory everywhere because all these "automated" migrations seem risky to me. Simply reusing the existingdefectdojo_redisvolume is fail-safe. Let me know your thoughts or alternative (simpler?) approaches to get rid of theredisin the volume name.UPDATE:- Easiest solution for us would be to instruct users to let all Celery tasks finish before shutting down the celeryworker(s). In that case we can start with a clean sheet with a nicely named volume (and Valkey 9).```