Skip to content

docker compose: switch to Valkey as message broker#13331

Merged
mtesauro merged 17 commits intoDefectDojo:devfrom
valentijnscholten:valkey-compose
Oct 31, 2025
Merged

docker compose: switch to Valkey as message broker#13331
mtesauro merged 17 commits intoDefectDojo:devfrom
valentijnscholten:valkey-compose

Conversation

@valentijnscholten
Copy link
Copy Markdown
Member

@valentijnscholten valentijnscholten commented Oct 5, 2025

Fixes #13323

Since the license change at Redis the fork ValKey has become widely popular and is backed by industry giants such as AWS. AWS is advising to use ValKey over Redis and is using lower prices for ValKey compared to Redis.

This PR migrates the docker compose setup to use Valkey instead of Redis.

The simplest approach is chosen:

  • re-use the existing Redis Docker volume

Justification

Since Valkey 7.2 is a fork of Redis 7.2 it is a drop-in replacement. I've tested the by generating a large backlog of celery task before making the changes in this PR. Shuttting down, applying the changes and bringin Defect Dojo up resulted in celeryworkers resuming work and processing the backlog.

I've checked the Dockerfiles for both Redis and Valkey. They show that they're laregely the same:

  • both use /data to store their data
  • both use id=999, gid=1000 for file ownership

The Valkey docs mention alternative migration paths such as taking snapshots and restoring these:

The re-use of the existing volume is much simpler and fits with the statement that Valkey 7.2 is compatible with Redis 7.2

In theory we could go to Valkey 8 or 9, but somehow it feels like the safer option to stick with 7.2 for now.

Client libraries

I tried migrating from the redis Python library to the valkey Python library, but ran into some issues with Celery. Research shows it's recommended to stick to the redis library unitl all other frameworks such as Celery have adopted first-class support of the Valkey platform.

Volume names still contains redis

By re-using the existing docker volume, we are stuck with the name defectdojo_redis. We can't rename volumes, so we just have to live with this. Or.....

EDIT: We've chosen to retain the volume and accept for now that it is named defectdojo_redis

1) Its possible to change the name of the volume to defectdojo_valkey if and only if users perform some manual steps during the upgrade. This would look something like:

~~ ~~`docker compose down`~~ ~~`docker volume create XXXX_defectdojo_valkey`~~ ~~`docker run --rm -u 0:0 -v oldvol:/from -v XXXX_defectdojo_valkey:/to alpine sh -c 'tar -C /from -cpf - . | tar -C /to -xpf -'`~~ ~~`docker compose up -d`~~ ~~`docker volume rm XXXX_defectdojo_redis`~~ ~~
The XXX is the prefix given to volume names by docker compose, usually based on the folder Defect Dojo is in.
In theory we could create a script that reads the exact volume name and have a migrate-redis-to-valkey.sh script that does everything transparently.

2) That script could (in theory) also be run by the initializer. Something like this could be added to the entrypoint:

~~ ~~# Optional one-time migration from old Redis volume (/redis-data) to new Valkey volume (/valkey-data)~~ ~~# Runs before DD_INITIALIZE check so it executes even if initialization is skipped~~ ~~if [ "${DD_MIGRATE_REDIS_VOLUME:-1}" != "0" ] && [ -d /redis-data ] && [ -d /valkey-data ]; then~~ ~~ if [ ! -f /valkey-data/.dojo_redis_migrated ] \~~ ~~ && [ -z "$(ls -A /valkey-data 2>/dev/null || true)" ] \~~ ~~ && [ -n "$(ls -A /redis-data 2>/dev/null || true)" ]; then~~ ~~ echo "Migrating data from old Redis volume (/redis-data) to new Valkey volume (/valkey-data)..."~~ ~~ tar -C /redis-data -cpf - . | tar -C /valkey-data -xpf -~~ ~~ touch /valkey-data/.dojo_redis_migrated~~ ~~ else~~ ~~ echo "Skip migration: destination not empty, source empty, or already migrated"~~ ~~ fi~~ ~~fi~~ ~~

3) Or we could add a one-time init service to the docker-compose file. Something like:

```
services:
~~ valkey-init-migrate:~~
~~ image: alpine:3.19~~
~~ command: >~~
~~ sh -c 'set -eu;~~
~~ if [ -z "$(ls -A /to 2>/dev/null)" ] && [ -n "$(ls -A /from 2>/dev/null)" ]; then~~
~~ tar -C /from -cpf - . | tar -C /to -xpf -;~~
~~ else~~
~~ echo "Skip copy: /to not empty or /from empty";~~
~~ fi'~~
~~ volumes:~~
~~ - oldredisdata:/from:ro~~
~~ - valkeydata:/to~~
~~ restart: "no"~~

~~ valkey:~~
~~ image: valkey/valkey:7.2-alpine~~
~~ depends_on:~~
~~ valkey-init-migrate:~~
~~ condition: service_completed_successfully~~
~~ volumes:~~
~~ - valkeydata:/data~~
~~ # command: ["valkey-server","--appendonly","yes"] # if you used AOF~~

volumes:
~~ oldredisdata:~~
~~ external: true # existing Redis volume name~~
~~ name: oldredisdata~~
~~ valkeydata:~~
~~ # new volume Compose will create (your new name)~~
```

I say in theory everywhere because all these "automated" migrations seem risky to me. Simply reusing the existing defectdojo_redis volume is fail-safe. Let me know your thoughts or alternative (simpler?) approaches to get rid of the redis in the volume name.

UPDATE:
- Easiest solution for us would be to instruct users to let all Celery tasks finish before shutting down the celeryworker(s). In that case we can start with a clean sheet with a nicely named volume (and Valkey 9).
```

@valentijnscholten valentijnscholten added this to the 2.52.0 milestone Oct 5, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 6, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 6, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 6, 2025

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 7, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 7, 2025

Conflicts have been resolved. A maintainer will review the pull request shortly.

@mtesauro
Copy link
Copy Markdown
Contributor

mtesauro commented Oct 7, 2025

For the record, I'm all for this approach. Keeping an 👁️ on this for it to leave Draft status 😉

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 7, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Oct 7, 2025

Conflicts have been resolved. A maintainer will review the pull request shortly.

@9alexx3
Copy link
Copy Markdown
Contributor

9alexx3 commented Oct 8, 2025

Hello @valentijnscholten

This migration fixes the Redis latest vulnerability (CVE-2025-49844) that was fixed in v8.2.2?

@valentijnscholten
Copy link
Copy Markdown
Member Author

Yes it does: https://github.com/valkey-io/valkey/releases/tag/7.2.11, but we'll be updating redis to 7.2.12 before we're moving to valkey as we still have some testing/adjustments to do to the helm chart.

@valentijnscholten
Copy link
Copy Markdown
Member Author

Looks there's some contradicting info floating around as according to Redis themselves the CVE is fixes in 7.2.11, which we are already running: https://github.com/redis/redis/releases/tag/7.2.11

@mtesauro
Copy link
Copy Markdown
Contributor

mtesauro commented Oct 8, 2025

Looks there's some contradicting info floating around as according to Redis themselves the CVE is fixes in 7.2.11, which we are already running: https://github.com/redis/redis/releases/tag/7.2.11

This was my understanding aka DefectDojo was not vulnerable when this was reported due to the version we had in the lastest version.

So, keep those DefectDojo instances up to the latest, greatest version 👍

@github-actions
Copy link
Copy Markdown
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Copy Markdown
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Copy link
Copy Markdown
Member Author

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion

Comment thread docs/content/en/open_source/upgrading/2.52.md Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

Updated documentation for version 2.52 to reflect the transition from Redis to Valkey as the message broker, including UI fixes and deduplication improvements.
@valentijnscholten valentijnscholten marked this pull request as ready for review October 27, 2025 18:13
@dryrunsecurity
Copy link
Copy Markdown

DryRun Security

This pull request includes docker-compose.yml with hardcoded default secrets (DD_SECRET_KEY and DD_CREDENTIAL_AES_256_KEY); while intended for development, leaving them in a file usable in production risks session compromise and credential decryption if not overridden. Consider removing defaults, loading secrets from environment or a secrets manager, or ensuring the compose file is strictly gated to non-production use.

Hardcoded Default Secrets in docker-compose.yml
Vulnerability Hardcoded Default Secrets
Description The docker-compose.yml file contains hardcoded default values for DD_SECRET_KEY and DD_CREDENTIAL_AES_256_KEY. While these are intended as defaults for development environments, their presence in a file that could be used for production deployments without explicit overrides poses a risk. If these values are not overridden, an attacker who knows these default keys could compromise session integrity and decrypt sensitive credentials.

DD_SECRET_KEY: "${DD_SECRET_KEY:-hhZCp@D28z!n@NED*yB!ROMt+WzsY*iq}"
DD_CREDENTIAL_AES_256_KEY: "${DD_CREDENTIAL_AES_256_KEY:-&91a*agLqesc*0DJ+2*bAbsUZfR*4nLw}"


All finding details can be found in the DryRun Security Dashboard.

@Maffooch Maffooch requested review from dogboat and rossops October 27, 2025 18:54
Copy link
Copy Markdown
Contributor

@mtesauro mtesauro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@mtesauro mtesauro merged commit 4fda41e into DefectDojo:dev Oct 31, 2025
152 checks passed
Maffooch pushed a commit to valentijnscholten/django-DefectDojo that referenced this pull request Feb 16, 2026
* docker compose: switch to valkey as message broker

* docker compose: switch to valkey as message broker

* docker compose: switch to valkey as message broker

* docker compose: switch to valkey as message broker

* docker compose: switch to valkey as message broker

* Update 2.52.md

* Update docs/content/en/open_source/upgrading/2.52.md

Co-authored-by: kiblik <5609770+kiblik@users.noreply.github.com>

* Update 2.52.md

* Update docs/content/en/open_source/upgrading/2.52.md

* Revise 2.52 upgrade notes for Valkey integration

Updated documentation for version 2.52 to reflect the transition from Redis to Valkey as the message broker, including UI fixes and deduplication improvements.

---------

Co-authored-by: kiblik <5609770+kiblik@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants