Skip to content

fix(cache): pass watch keys to Transaction so optimistic lock actually works#701

Merged
fatelei merged 4 commits into
mainfrom
feat/redis-cluster-support
Apr 24, 2026
Merged

fix(cache): pass watch keys to Transaction so optimistic lock actually works#701
fatelei merged 4 commits into
mainfrom
feat/redis-cluster-support

Conversation

@lin-snow

@lin-snow lin-snow commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

Description

cache.Transaction(fn) used to call client.Watch(ctx, fn) without passing any keys, so it ran a "MULTI/EXEC without optimistic lock" — the double-write of id2key / key2id in debugging_service didn't actually guard against concurrent races.

  • Added variadic signature: Transaction(fn, watchKeys ...string). The varargs default to nil, so existing Transaction(fn) call sites are byte-level backward compatible.
  • Keys are passed through serialKey() to apply the prefix before handing off to go-redis, matching other helpers.
  • Migrated debugging_service/connection_key.go to the new signature and WATCH on id2key: concurrent creation of a connection key for the same tenant now relies on WATCH retry for correctness instead of leaning on the inner SetNX's NX semantics.

Bundled helper extraction:

  • New pkg/utils/parser/csv.go SplitAndTrimCSV ("a, b , ,c"["a", "b", "c"], empty string returns nil), with csv_test.go covering the edge cases.
  • The sentinel branch in plugin_manager/manager.go switched from strings.Split(configuration.RedisSentinels, ",") to this helper, so accidental whitespace in REDIS_SENTINELS is parsed correctly.

Type of Change

  • Bug fix (fake-transaction semantics in Transaction)
  • Refactor (CSV parsing extraction)
  • New feature
  • Performance improvement
  • Other

Essential Checklist

Testing

  • go build ./internal/... ./pkg/... passes; new unit tests in parser/csv_test.go are green; existing cache / debugging_service tests did not regress.
  • debugging_service integration smoke test (local Redis Sentinel) is pending.

Bug Fix (if applicable)

  • N/A (no external issue tracking, self-discovered).

Additional Information

The variadic change is source-compatible — no existing Transaction(fn) call site needs to change. The new watchKeys parameter is optional; callers that want WATCH just pass keys positionally.

@lin-snow lin-snow self-assigned this Apr 18, 2026
@lin-snow lin-snow added the enhancement New feature or request label Apr 18, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds Redis Cluster support, including new configuration fields and the InitRedisClusterClient utility. The PluginManager now supports cluster initialization. Key feedback highlights that the current Transaction() implementation is incompatible with Redis Cluster, suggests generalizing error messages in utility functions, and recommends refactoring redundant TLS configuration logic.

Comment thread pkg/utils/cache/redis.go Outdated
Comment thread pkg/utils/cache/redis.go Outdated
Comment thread pkg/utils/cache/redis.go Outdated
@lin-snow lin-snow changed the title feat(redis): add Redis Cluster client support to plugin-daemon fix(cache): pass watch keys to Transaction so optimistic lock actually works Apr 22, 2026
lin-snow and others added 4 commits April 22, 2026 17:10
Wire the missing third branch so operators can point the daemon at a
Redis Cluster without falling back to dialing an empty REDIS_HOST:PORT
and panicking at startup.

* introduce REDIS_USE_CLUSTERS / REDIS_CLUSTERS /
  REDIS_CLUSTERS_PASSWORD envs, aligned with dify api naming so Helm
  can set a single env group
* branch in PluginManager.Launch for Cluster ahead of the existing
  Sentinel / standalone paths; falls back to REDIS_PASSWORD if the
  cluster-specific password is not set
* implement cache.InitRedisClusterClient via redis.NewClusterClient;
  downstream cache / lock / pub-sub helpers keep working because
  `client` is declared as redis.UniversalClient

Redis Cluster disables SELECT DB, so the cluster branch does not
plumb RedisDB. This is intentional and documented in the release
note accompanying the overall Redis Cluster support work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Narrow the plugin-daemon Redis client to standalone and sentinel modes;
remove the cluster code paths that were briefly exercised on this branch.

Changes:
- Remove REDIS_USE_CLUSTERS / REDIS_CLUSTERS / REDIS_CLUSTERS_PASSWORD
  env fields from app.Config.
- Remove the cluster branch (plus the two cluster-specific fail-fast
  guards on REDIS_USE_CLUSTERS + REDIS_USE_SENTINEL and REDIS_DB != 0)
  from PluginManager.Launch. The init block is now just sentinel or
  standalone.
- Delete cache.InitRedisClusterClient. The main redis client + all
  helpers (Transaction, pub/sub, lock, etc.) already route through
  redis.UniversalClient so downstream code needs no adjustment.

Intentionally kept:
- Transaction(fn, watchKeys...) signature: the variadic signature is
  backwards-compatible with every existing Transaction(fn) call and the
  one new-style caller (debugging_service) benefits from real WATCH
  semantics even on standalone — this is correctness-independent of
  cluster support.
- parser.SplitAndTrimCSV: the sentinel branch migrated to it in 617e7a5
  and still uses it to parse REDIS_SENTINELS, so the helper has a
  standalone/sentinel consumer and stays.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lin-snow lin-snow force-pushed the feat/redis-cluster-support branch from e57d7af to 4dd87ac Compare April 22, 2026 09:14
@lin-snow lin-snow marked this pull request as ready for review April 22, 2026 09:23
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Apr 22, 2026
@lin-snow lin-snow requested a review from wylswz April 22, 2026 09:23
@dosubot dosubot Bot added bug Something isn't working go Pull requests that update go code labels Apr 22, 2026
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Apr 24, 2026
@fatelei fatelei merged commit 98b24b5 into main Apr 24, 2026
7 checks passed
@fatelei fatelei deleted the feat/redis-cluster-support branch April 24, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request go Pull requests that update go code lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants